Sensor
Note
Most Sensor properties are set via JVM arguments; the file name is listed when the property can be set in a file.
Specified by adding -D<propertyName>=<propertyValue> to the list of JVM args, e.g.-Dclient.rest.conntimeout.ms=0
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.client.rest.queue The queue length for outgoing REST HTTP requests. | 20000 | ||
com.unraveldata.client.rest.retryfail The cool-down period between unsuccessful attempts to complete a REST HTTP request. | ms | 30 sec | |
com.unraveldata.client.rest.conn.timeout.ms REST HTTP request timeout. | ms | 100 | |
com.unraveldata.client.rest.shutdown.ms Maximum time to wait for orderly shutdown of the REST client (if exceeded some messages still in the queue will be lost). | ms | 10 | |
com.unraveldata.client.rest.dns.ttl The period to refresh the DNS information - IP is pre-resolved and kept until the next refresh if no failures are observed. | ms | 6 hours | |
com.unraveldata.client.rest.priority.retries Certain critical messages have a priority flag, and this is the number of retries upon failed transmission. | 5 | ||
unravel.server.hostport Unravel server host:port information. | - | ||
com.unraveldata.client.resolve.hostname If set to | boolean | true |
Specified in hive-conf, or in the Hive CLI or beeline session. (CDH 5.14 only.)
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.com.hive.spark.enabled Set to true to allow Hive and Spark applications to be linked. Only available for CDH 5.14. | boolean | true |
Specified by adding -D<propertyName>=<propertyValue> to the list of JVM args, e.g.-Dclient.rest.conntimeout.ms=0
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.agent.metrics.enabled_keys Comma separated list of metric type names which are enabled for collection. | CSL | availableMemory,cpuUtilization,processCpuLoad,systemCpuLoad,maxHeap,usedHeap,vmRss,gcLoad | |
unravel.metrics.factor Sampling period scale down factor. | 1 |
Agent arguments are added directly to the agent definition - anything after -javaagent:btrace-agent.jar= is considered to be the agent argument. The arguments are delimited by comma with no extra space.
Agent argument/definition | Set by user | Unit | Default |
|---|---|---|---|
metricsCaptureFilter Format allow specifying single ordinals for component IDs as well as ranges and enumerations. Fore example: metricsCaptureFilter=1,2,5-10,turns on metrics collection for components 1, 2, 5 to 10. | 0-1500 |
Specified in spark-defaults.conf .
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
spark.unravel.shutdown.delay.ms Amount of time to delay shutdown so the last messages are processed (allows Btrace sensor to send all the data before the spark driver exits). | ms | 30 |
Specified by adding -D<propertyName>=<propertyValue>; to the list of JVM args, e.g.-Dclient.rest.conntimeout.ms=0
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.spark.sensor.enableCachingInfo Enable tracking caching info for Spark apps. | boolean | False | |
com.unraveldata.spark.sensor.enableSampling Enable data sampling between operators for Spark apps. | boolean | False | |
spark.unravel.program.path hdfs/S3/adl path to the source zip file of the Spark app. This is a new way to upload spark program. Unravel daemons can access this file. |
These properties can be specified either in the spark-defaults.conf file or in the spark-submit command while running the Spark app.
Property/Description | Set by User | Unit | Default |
|---|---|---|---|
spark.unravel.metrics.level
Valid range: 1-5 | integer | 3 | |
spark.unravel.sql.updates.enabled Enables or disable SQL updates from sensor.
| boolean |
| |
spark.unravel.listener.max.task.count The number of tasks kept in memory by UnravelListener and sent to Unravel daemon when spark.unravel.metrics.level =4. Valid values: >= 0 | integer | 100 | |
spark.unravel.live.update.interval.sec The frequency of updates form sensor. The updates are sent only on task completion, Stage completion, Job start and on job end events. Valid range: >= 0 | integer | 60 | |
spark.unravel.sensor.log.level Controls the sensor messages for easy debugging. Valid values: debug, info, warn, error | Member of set | error |