com.unraveldata.auto.action.enable.policy.enforce.in.jcs

Enables AutoActions to be processed under the legacy (pre-4.5.2.0) mode of operation when AutoAction enforcement resided in JCS2 daemon. This works for on-prem mode only. When using this mode you can stop AutoAction service.

com.unraveldata.auto.action.max.buffered.group.count

A message group represents one complete YARN metrics polling cycle. It is incomplete when it has missing messages, messages out of order, etc. This property controls how many message groups can be buffered while waiting for completion of the current group. Once the daemon has buffered the max.buffered.group.count , it either drops or accepts the incomplete group (see max.lost.messages.count ) and moves onto processing the next group.

Setting this value higher than 1 may increase latency of enforcement of an AutoActions but can help to alleviate errors in message transport protocol.

com.unraveldata.auto.action.max.lost.messages.count

Maximum number of lost messages a group can have and still be accepted for aggregation and policy enforcement.

Accepting incomplete groups lowers the consistency of triggered violations but lets the daemon to operate with an “unstable” connection.

0: only complete groups are accepted.

> 0: groups missing up to X messages are accepted.

com.unraveldata.auto.action.metric.discard.ms

When an incoming metric's timestamp is older than this value, it isn't processed but is discarded.

This mechanism is designed to deal with Kafka latency, consumer lags and other message delivery delays. It prevents the daemon from acting on outdated data and issuing false-positive violation events that are irrelevant at the present time

com.unraveldata.auto.action.policy.db.update.skip.cycle

Defines how often the daemon refreshes the AutoAction policy definitions when aggregating and enforcing per enforcement cycle (see enforce.period.ms ).

0: All policies are read every time the daemon's ready to aggregate and enforce policies.

> 0: The policies are read every X cycle. For instance, 1: the daemon reads them every other cycle, 2: every third period, and so on.

Changing this value makes sense only if there are a lot of policies defined.

com.unraveldata.auto.action.policy.enforce.period.ms

Maximum wait period for policy enforcement to be triggered.

AutoAction metric aggregation and enforcement cycle is driven by metric producers, i.e., the daemon synchronizes with the metric polling cycles for each monitored cluster.

When no YARN metrics are delivered or accepted no policies are being evaluated for longer than this period, the daemon forcibly executes metric aggregation and policy enforcement to process other internal Unravel metrics, such as Workflow, Hive, Tez, Impala metrics.

com.unraveldata.auto.action.transport.receiver.ttl.ms

Identifies how long a transport protocol connection remains active without receiving any data from the cluster. If no metrics are received for longer than this value, the cluster is considered terminated and the connection is closed on the receiver (daemon's) side.

com.unraveldata.job.collector.running.load.conf

This is required for Hive metrics in running state.