com.unraveldata.com.hive.spark.enabled

Set to true to allow Hive and Spark applications to be linked.

Only available for CDH 5.14.

Specified in hive-conf, or in the Hive CLI or beeline session. (CDH 5.14 only.)

true

com.unraveldata.spark.live.interval.sec

This is the interval in seconds after which live application data is updated. It allows for tracking of Spark tasks. The Spark APM updates on Task completion in addition Job start, and Job and Stage completion.

This is a sensor property and is not updated unravel.properties.

60

com.unraveldata.mr.parse.input.split

Controls whether to parse splits information for map reduce jobs from YARN aggregated logs or not.

true

com.unraveldata.job.collector.log.aggregation.base

HDFS path to the aggregated container logs (logs to process). Do not include the hdfs://prefix. The log format defaults to TFile. You can specify multiple logs and log formats (TFile or IndexedFormat.)

Example: com.unraveldata.job.collector.log.aggregation.base=TFile:/tmp/logs/*/logs/,IndexedFormat:/tmp/logs/*/logs-ifile/.

/tmp/logs/*/logs/

com.unraveldata.action.killmove.enabled

Allows users to kill and move apps.

true: users can kill and move appls

false: only admins can kill and move apps.

false

com.unraveldata.ext.kafka.aggs.broker

100

com.unraveldata.ext.kafka.aggs.consumer_group

1000

com.unraveldata.ext.kafka.aggs.broker_metrics

1000

airflow.look.back.num.days

The number of days to look back. The look-back days can be specified as a positive or negative number. For instance -5 or 5 sets the look back number of days to 5.

airflow.look.back.num.hours

The look-back time window in hours, which can be either a positive or negative integer. If present, it takes precedence over airflow.look.back.num.days to have finer granularity. Suggested value for large clusters: 2

com.unraveldata.airflow.task.log.parsing.enabled

Controls whether to parse the Airflow Task logs. These logs are used to populate the Workflow Instance entities in the "Jobs - Workflows" page.

com.unraveldata.airflow.task.log.parsing.operators

Controls the Task logs to parse based on the Operator that produced it. Since Unravel only derives insights for Hive, Spark, and MR applications, it is set to only analyze operators that can launch those types of jobs.

The values are delimited using a "," and it treats "*" as a wildcard to many any or no characters. Any special characters like "\" or "." will be removed.

com.unraveldata.airflow.task.thread.pool.size

Controls whether to process the Airflow Task logs sequentially or in parallel. Process the logs in parallel improves performance. This config takes effect only if airflow.task.log.parsing.enabled=true.

Possible values are 1-16. If one, logs are processed sequentially. If greater than one, logs are processed in parallel using a Thread Pool with of the size specified value.

com.unraveldata.session.dynamicAllocation.enabled

If set to true Spark sessions recommends dynamic allocation. If users don't want that they need to explicitly set it to false.

com.unraveldata.session.spark1.submit.command

The command to use for Spark1 applications while applying the session run. This path should be fully qualified.

com.unraveldata.session.spark2.submit.command

The command to use for Spark2 applications while applying the session run. This path should be fully qualified.

com.unraveldata.spark.shutdown.delay.ms

Amount of time to delay shutdown so the last messages are processed (allows Btrace sensor to send all the data before the spark driver exits).

com.unraveldata.job.collector.running.load.conf

When set to true

com.unraveldata.job.collector.hive.queries.cache.size

This is used to improve the Hive-MR pipeline by caching data so it can be retrieved from cache instead of external API. You should not have to change this value.

com.unraveldata.auto.action.metric.discard.ms

The look-back time window in hours, which can be either a positive or negative integer. If present, it takes precedence over airflow.look.back.num.days to have finer granularity. Suggested value for large clusters: 2

3600000

com.unraveldata.ldap.mailAttribute

The mail attribute name in the LDAP response that Unravel server will use to extract the ldap user's email address.

If not configured, Unravel server use use the attribute name "mail".

-

com.unraveldata.ldap.customLDAPQuery

replaced hive.server2.authentication.ldap.customLDAPQuery

-

com.unraveldata.ldap.groupFilter

replaced hive.server2.authentication.ldap.groupFilter

-

com.unraveldata.ldap.groupDNPattern

replaced hive.server2.authentication.ldap.groupDNPattern

-

com.unraveldata.ldap.guidKey

replaced hive.server2.authentication.ldap.guidKey=uid

-

com.unraveldata.ldap.userDNPattern

replaced hive.server2.authentication.ldap.userDNPattern

-

com.unraveldata.ldap.userFilter

replaced hive.server2.authentication.ldap.userFilter

-

com.unraveldata.ldap.groupMembershipKey

replaced hive.server2.authentication.ldap.groupMembershipKey

-

com.unraveldata.ldap.groupClassKey

replaced hive.server2.authentication.ldap.groupClassKey

-

com.unraveldata.custom.banner.display

Displays a banner at the top of the Unravel UI.

true: banner displays text until end.date

false: no change to UI

false

com.unraveldata.custom.banner.text

Text to display when display is true

-

com.unraveldata.custom.banner.end.date

Date and Time to stop displaying the custom banner

Format: YYYYMMDDTHHMMSSZ-000000

-

com.unraveldata.hbase. clustername .node.http.apis

HBase node web UI.

Format: http[s]://host:port,http[s]://host:port,... *

Example: http://your.master.server:16010,http://your.region.server:16030

-

com.unraveldata.live.logreceiver.port.https

HTTPS server port (negative value means disabled HTTPS server)

-1

com.unraveldata.server.ssl.cert_path

KeyStore file path. For example, /usr/local/unravel/cert.jks

-

com.unraveldata.server.ssl.cert_password

KeyStore password

-

com.unraveldata.server.ssl.trust_store_path

TrustStore file path

-

com.unraveldata.server.ssl.trust_store_password

TrustStore password

-

unravel.python.files_use_avg_file_size_flag

true: average of all the files is used against the threshold criteria and either all the files are accepted/counted or rejected/not counted as per the criteria.

false: absolute file size is used against the threshold criteria and a file is accepted/counted or rejected/not counter as per the criteria.

-

unravel.python.min_parent_dir_depth

Directory depth to start search at.

-

unravel.python.max_parent_dir_depth

Directory depth to end search at. Maximum is 50.

-

unravel.python.drill_down_subdirs_flag

When set a file is accounted (listed) for all its ancestors.

false: a file is accounted in only its immediate parent. This allows Unravel to find a specific directory with maximum number of files matching the size criteria.

-

unravel.python.reports.files.small_files_use_avg_file_size_flag

true

unravel.python.reports.files.small_files_min_parent_dir_depth

0

unravel.python.reports.files.small_files_max_parent_dir_depth

10

reports.files.small_files_drill_down_subdirs_flag

true

The following four properties are defined per file Size: huge, medium, tiny, small, empty

unravel.python.reports.files. Size _files_use_avg_file_size_flag

false

unravel.python.reports.files. Size _file_min_parent_dir_depth

0

unravel.python.reports.files. Size _file_max_parent_dir_depth

10

unravel.python.reports.files. Size _file_drill_down_subdirs_flag

false

com.unraveldata.ambari.manager.url

URL of Cloud Manager. For example, http://$clouderaserver:7180, http://$ambariserver:8080

For Cloudera, if the URL does not contain a port you must define manager.port below

-

com.unraveldata.ambari.manager.username

Username to log into the manager

-

com.unraveldata.ambari.manager.password

Password for the username.

-

com.unraveldata.spark.master

Default Spark master mode to be used if not available from the sensor.

-

com.unraveldata.spark.eventlog.appDuration

If application duration is more than configured value load the event log is not loaded.

1440 mins

com.unraveldata.appstatus.refresh.mins

Time interval in minutes to scan for running applications and marking the stale ones.

5 min

com.unraveldata.appstatus.stale_limit.mins

Maximum number of minutes since the latest app update from RM before it is marked as stale.

Unit: minutes

10

com.unraveldata.cluster_access.host

Cluster Access Service host (where the service is bound).

0.0.0.0

com.unraveldata.cluster_access.port

Cluster Access Service port.

4020

com.unraveldata.sregistry.hostport

Service Registry host:port.

${com.unraveldata.zk.quorum}

com.unraveldata.sensor.polling.secs

The base polling period of Unravel reactive sensors in seconds.

30s

com.unraveldata.appevents.emitters.exclude.list

Comma separated list of the application event emitter IDs which will be disabled/excluded.

-

com.unraveldata.multicluster.enabled

Allow Unravel to operate in multi-cluster mode.

In this mode a service registry will be used to discover and access all registered (local and remote) clusters.

false