What is tagging?
Tagging is a method to apply more context to your applications so you can view and organize them with more specificity based upon your requirements and needs.
There are two types of tags:
Application tags provide ways to filter applications.
Workflow tags logically group a set of applications.
Application tagging
By the time your application reaches the cluster much of its context is lost, e.g., the dept which submitted it, what project it belongs to, etc.
All applications have the following metadata. (See how the metadata fields are populated.)
App type
App Id
App Name
Username
Cluster (empty for server-less applications)
Queue (empty for server-less applications)
Configuration (accessible for some applications)
As an admin you need to be able to organize/view the applications running on the cluster with more granularity than this limited set allows. You will want to “slice and dice" applications a myriad of ways for a variety of reasons.
Unravel can not “magically” deduce which applications belong to what tenants, projects, teams, etc. to gain such granularity you must provide Unravel with this information.
Tagging an app is how you tell Unravel about its context.
What is a tag
At the basic level a tag is simply a <key, value> pair which is associated with an application.
A simple way to conceptualize tagging is to think of a spreadsheet. The key is the column header and the value an entry in the column. For each application you enter your value, if any, in the column. The following example has two keys, Tenant and Project, which can be used to filter applications. While an app can have multiple tags, it cannot have multiple values for a given key. In this example, an app cannot be tagged with proj1 and proj2,
App Type | App ID | App Name | User | Cluster | Queue | Tenant | Project |
|---|---|---|---|---|---|---|---|
MR | j_134 | distc | John | def | root.UM-proj1.print | marketing | proj1 |
Spark | s_345 | rate.spark.sim | Jane | def | root.UM-proj2.print | sales | proj2 |
Spark | s_456 | over.spark.sim | Jane | def | root.UM-proj3.print | sales | proj3 |
See the following Example for an explanation of how these tags were generated.
Effective use of tags
You can use tags to:
Generate chargeback reports based upon specific criteria, e.g., project, dept, team, etc.
Decide which apps a specific user can see, e.g., the marketing head can see all marketing apps, while user2 can only see specific marketing project apps.
Group applications together. (See Tagging workflows.)
In order to effectively use tags you need to understand your requirements for displaying or grouping your apps. For example:
Do you need chargebacks reports for each tenant in a multi-tenant cluster? Then apps must be tagged with the tenant it belongs to.
Do applications need to be billed back to departments and teams? Apps need department and team tags.
Do you want to allow some users to see all projects and others just a subset of the projects (see Role Based Access Control)? Apps must have the project tags.
A specific tag can be used for different purposes. You can use the project tag to generate chargeback reports and filter views specific users.
Assigning tags
You generate a tagging dictionary via a Python script. Unravel then uses the dictionary to apply tags to the applications as you prescribe.
But like Unravel, the only information you have about the application running on the cluster is its metadata. So how can you develop a script to tag specific applications; how can you determine and generate your <key, value> pairs?
Methods to generate/create tags for an application
By creating naming conventions for app, queues, and cluster names you can embed information to use for your tags, e.g., placing all apps belonging to project-1 in the root.UM-proj1.print queue lets you extract the project name from the queue the app is in.
You can use the app’s metadata to create the tag values, for example:
Directly, e.g., <team,username>
Parse it to extract information, e.g., <project, {extracted fromqueue name}>
Concatenate metadata with other metadata or strings, e.g., <dept, {username}+ {extracted fromapp name} + string>
A tagging script can access files that contain further mapping information. e.g., maps projects to tenants.
You can download example tagging scripts. (This is currently private; please contact Unravel Support.)
Example
The above table is simple example to help you understand tagging concepts. In your environment you will likely use a more complicated schema. In this example we show how the tags were created.
The cluster is multi-tenant; we created the tags:
<tenant, marketing>
<tenant, sales>
There are three projects; we created the tags:
<project, proj1>
<project, proj2>
<project, proj3>
Finally, we have a file that maps the projects to the tenants:
proj1 to marketing
proj2 and proj3 to sales
We then told Unravel about the tags and how to assign them, i.e., developed the tagging dictionary.
First, an app’s queue is parsed to extract the project it belongs to. The project name is encoded between "UM-" and ".". Once extracted, the name was used as the project value. Next, the script accessed a file which mapped the project name (value) to the tenant (key).
The script resulted in three applications being assigned tenant and project tags.
Workflow tagging
Workflow tags are much simpler than application tags. You use preexisting Unravel tags to create workflows, specifically unravel.workflow.name and unravel.workflow.utctimestamp. See Tagging workflows for more information on creating workflows.