com.unraveldata.airflow.available

Notes if the airflow is currently available.

false: not available

true: available

airflow.look.back.num.days

The number of days to look back. The look-back days can be specified as a positive or negative number. For instance -5 or 5 sets the look back number of days to 5.

airflow.look.back.num.hours

The look-back time window in hours, which can be either a positive or negative integer. If present, it takes precedence over airflow.look.back.num.days to have finer granularity. Suggested value for large clusters: 2

com.unraveldata.airflow.http.max.body.size.byte

Set maximum number of bytes Unravel fetches data from Airflow Web UI. Default unlimited.

com.unraveldata.airflow.login.name

Airflow UI login username. You must set this if airflow.server.url = https.

com.unraveldata.airflow.login.password

Password for Airflow UI com.unraveldata.airflow.login.name. You must set set this if airflow.login.name is set.

com.unraveldata.airflow.protocol

Type of connection, e.g., HTTPS or HTTP. You must set the airflow.login.name and airflow.login.password when this value is https.

com.unraveldata.airflow.server.url

Full URL of the airflow server, starting with http:// and https://.

com.unraveldata.airflow.status.timeout.sec

Set Airflow workflow status timeout in Unravel.

com.unraveldata.airflow.task.log.parsing.enabled

Controls whether to parse the Airflow Task logs. These logs are used to populate the Workflow Instance entities in the "Jobs - Workflows" page.

com.unraveldata.airflow.task.log.parsing.operators

Controls the Task logs to parse based on the Operator that produced it. Since Unravel only derives insights for Hive, Spark, and MR applications, it is set to only analyze operators that can launch those types of jobs.

The values are delimited using a "," and it treats "*" as a wildcard to many any or no characters. Any special characters like "\" or "." will be removed.

com.unraveldata.airflow.task.thread.pool.size

Controls whether to process the Airflow Task logs sequentially or in parallel. Process the logs in parallel improves performance. This config takes effect only if airflow.task.log.parsing.enabled=true.

Possible values are 1-16. If one, logs are processed sequentially. If greater than one, logs are processed in parallel using a Thread Pool with of the size specified value.