Home

Monitoring individual Hive queries

The Unravel JVM sensor is a prepackaged distribution of JVM agent which enables the collection of additional information, including resource usage metrics. The sensor binary is distributed as unravel-agent-pack-bin.zip.

Note

unravel-host is your Unravel gateway server. Port 4043 is where Unravel LR server is running.

When you enable this sensor, it uses the standard MapReduce profiling extension. You can copy and paste the following configuration snippets for a quick bootstrap

Option A: Installing the MapReduce JVM sensor on CDH

  1. Enable profiling:

    set mapreduce.task.profile=true;

  2. Select map and reduce profiles:

    set mapreduce.task.profile.maps=0-5;
set mapreduce.task.profile.reduces=0-5;

  3. Enable the JVM agent for map and reduce tasks:

    set mapreduce.task.profile.params=- javaagent:/opt/cloudera/parcels/UNRAVEL_SENSOR/lib/java/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;

  4. Enable the JVM agent for application master:

    set yarn.app.mapreduce.am.command-opts=-    javaagent:/opt/cloudera/parcels/UNRAVEL_SENSOR/lib/java/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;
Option B: Installing the MapReduce JVM sensor on Cloudera Manager for Hive

See Optional - Configure YARN - MapReduce (MR) JVM Sensor Cluster-Wide in Step 2: Install Unravel Sensor and Configure Impala.

Option C: Installing the MapReduce JVM sensor on HDFS

Change your *init file as follows:

  1. Set the path to the JVM sensor archive.

    set mapreduce.job.cache.archives=path_in_hdfs/unravel-agent-pack-bin.zip;

  2. Enable profiling:

    set mapreduce.task.profile=true;

  3. Select map and reduce profiles:

    set mapreduce.task.profile.maps=0-5; 
set mapreduce.task.profile.reduces=0-5;

  4. Enable the JVM agent for map and reduce tasks:

    set mapreduce.task.profile.params=-javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;

  5. Enable the JVM agent for application master:

    set yarn.app.mapreduce.am.command-opts=-javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;
Option D: Installing the MapReduce JVM sensor on the local file system

  1. Add the JVM sensor archive to the PATH environment variable.

  2. Enable profiling:

    set mapreduce.task.profile=true;
set mapreduce.task.profile=true;
set mapreduce.task.profile=true;

  3. Select map and reduce profiles:

    set mapreduce.task.profile.maps=0-5; set mapreduce.task.profile.reduces=0-5;
set mapreduce.task.profile.maps=0-5; set mapreduce.task.profile.reduces=0-5;
set mapreduce.task.profile.maps=0-5; set mapreduce.task.profile.reduces=0-5;
set mapreduce.task.profile.maps=0-5; set mapreduce.task.profile.reduces=0-5;

  4. Enable the JVM agent for map and reduce tasks:

    set mapreduce.task.profile.params=-javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;

  5. Enable the JVM agent for application master:

    set yarn.app.mapreduce.am.command-opts=-javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043;
In this section: