Skip to main content



This topic explains how to configure Unravel to retrieve additional data from Hive, Tez, Spark and Oozie, such as Hive queries, application timelines, Spark jobs, YARN resource management data, and logs. You'll do this by generating Unravel's JARs and distributing them to every node that runs queries in the cluster. Later, after JARs are distributed to the nodes, you'll integrate Hive, Tez, and Spark data with Unravel.

1. Generate and distribute Unravel's Hive Hook and Spark Sensor JARs
2. Configure Ambari to work with Unravel
  1. Hive configurations

    1. Hive

      Click Hive > Configs > Advanced > Advanced hive-env. In the hive-env template, towards the end of line, add:

      export AUX_CLASSPATH=${AUX_CLASSPATH}:/usr/local/unravel-jars/unravel-hive-1.2.0-hook.jar 
    2. Hive Hook

      In Ambari's general properties, append ,com.unraveldata.dataflow.hive.hook.UnravelHiveHook, to the following properties:


      Be sure to append with no space before or after the comma, for example, property=existingValue,newValue


      For example, com.unraveldata.dataflow.hive.hook.UnravelHiveHook

    3. Custom

      In Ambari's custom hive-site editor set to unravel-gateway-internal-IP-hostname

      For example,

    4. Optional: Hive LLAP if it is enabled


      Edit hive-site.xml manually, not through Ambari Web UI.

      1. Copy the settings in Custom hive-interactive-site and paste them into /etc/hive/conf/hive-site.xml.

      2. Copy the settings in Advanced hive-interactive-env and paste them into /etc/hive/conf/hive-site.xml.


    If you have an Unravel version older than, create HDFS Hive Hook directories for Unravel:

    hdfs dfs -mkdir -p /user/unravel/HOOK_RESULT_DIR
    hdfs dfs -chown unravel:hadoop /user/unravel/HOOK_RESULT_DIR
    hdfs dfs -chmod -R 777 /user/unravel/HOOK_RESULT_DIR
  2. Configure HDFS.

    Click HDFS > Configs > Advanced > Advanced hadoop-env. In the hadoop-env template, look for export HADOOP_CLASSPATH and append Unravel's JAR path as shown.

    export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:<Unravel installation directory>/unravel-jars/unravel-hive-1.2.0-hook.jar
  3. Configure the BTrace agent for Tez

    In the tez-site.xml configuration file, append the Java options below to and tez.task.launch.cmd-opts:

    -javaagent:<Unravel installation directory>/unravel-jars/btrace-agent.jar=libs=mr,config=tez -Dunravel.server.hostport=unravel-host:4043


    In a Kerberos environment you need to modify property with the "run as" user or *.

  4. Configure the Application Timeline Server (ATS)


    From Unravel v4.6.1.6, this step is not mandatory.

    1. In yarn-site.xml:

      yarn.timeline-service.version=1.5 or yarn.timeline-service.versions=1.5f,2.0f
    2. If yarn.acl.enable is true, add unravel to yarn.admin.acl.

    3. In, add:

      Use ATS Logging: true
    4. In tez-site.xml, add:

      tez.history.logging.service.class=org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService"run-as"-user or *


      From HDP version 3.1.0 onwards, this Tez configuration must be done manually.

  5. Configure Spark-on-Yarn


    For unravel-host, use Unravel Server's fully qualified domain name (FQDN) or IP address.

    1. Add the location of the Spark JARs.

      Click Spark > Configs > Custom spark-defaults > Add Property and use bulk.png Bulk property add mode, or edit spark-defaults.conf as follows:


      • If your cluster has only one Spark 1.X version, spark-defaults.conf is in /usr/hdp/current/spark-client/conf.

      • If your cluster is running Spark 2.X, spark-defaults.conf is in /usr/hdp/current/spark2-client/conf.

      This example uses default locations for Spark JARs. Your environment may vary.

    2. Enable Spark streaming.

  6. Configure Oozie

3. Configure the Unravel Host

Define the following properties in <Unravel installation directory>/data/conf/ If you do not find the properties add them.

  1. Tez.

  2. Set these if the Application Timeline Server (ATS) requires authentication.

4. Optional: Confirm that Unravel UI shows Tez data.
  1. Run <Unravel installation directory>/install_bin/ on the HDP cluster or on any cloud environment where hive.execution.engine=tez.

  2. Log into Unravel server and go to the Applications page. Check for Tez jobs.

    Unravel UI may take a few seconds to load Tez data.

Adding a new node in an existing HDP cluster monitored by Unravel
1. Generate and distribute Unravel's Hive Hook and Spark Sensor JARs
2. For Oozie, copy the Hive Hook and BTrace JARs to the HDFS shared library path
3. If you have changed your Kerberos tokens or principal you must perform the following steps: