Skip to main content

Home

Tagging workflows
About Unravel workflow tags

You can add two Unravel tags (<key, value> pairs) to mark queries and jobs that belong to a particular workflow:

  • unravel.workflow.name: a string that represents the name of the workflow. The recommended format is TenantName-ProjectName-WorkflowName.

  • unravel.workflow.utctimestamp: a timestamp in yyyyMMddThhmmssZ format representing the logical time of a run of the workflow in UTC/ISO format. In UNIX/LINUX bash. You can get a timestamp in UTC format by running the command "$(date -u '+%Y%m%dT%H%M%SZ')".

    Note

    Do not put quotes ("") or blank spaces in/around the tag keys or values. For example:

    • SET unravel.workflow.name="ETL-Workflow; [Incorrect syntax]

    • SET unravel.workflow.name=ETL-Workflow; [Correct syntax]

Different runs of the same workflow have

  • The same value for unravel.workflow.name but

  • different values for unravel.workflow.utctimestamp.

Different workflows have different values for unravel.workflow.name.

Hive query example

This is a Hive query that was marked as part of the Financial-Tenant-ETL-Workflow workflow that ran on February 1, 2016:

SET unravel.workflow.name=Financial-Tenant-ETL-Workflow;
SET unravel.workflow.utctimestamp=20160201T000000Z;
SELECT foo FROM table WHERE … Your Hive Query text goes here
Easy recipes for tagging workflows

  • Export the workflow name and UTC timestamp from your top-level script that schedules each run of the workflow.

    Here, we use bash's date command to generate the timestamp.

    export WORKFLOW_NAME=Financial-Tenant-ETL-Workflow export UTC_TIME_STAMP=$(date -u '+%Y%m%dT%H%M%SZ')
Finding pipelines in Unravel web UI

Once your tagged workflows have been run, log into Unravel Web UI and select Jobs > Pipeline to start exploring Unravel's Workflow Management features.

In this section: