The Apps Running and Apps Pending count is not correct on the Operations page. (DT-119)

Applications details view

App Name value appears as “Databricks Shell” (class name instead of the last run_name) when jobs are submitted as Spark jar tasks, Notebook tasks and Python tasks. (DT-179, DT-176)

In some cases the Spark SQL query text and query plan mismatch. (DT-186)

When there are driver side exceptions in spark-submit application, the status is still captured as SUCCESS. It is difficult to capture driver side exceptions because they are not being logged in driver log by Databricks. (DT-192)

When a Python script is executed as a spark-submit task with eventLog enabled it becomes stuck forever. This is a bug at Databricks end which we have raised in the databricks forum.

Workaround is to not enable eventLog for spark-submit task which runs the Python script.

Filter by Application Name has issues when it has special characters like hyphen etc. By default, the spark.app.name has hyphens (e.g. app-20190712214639-0000). So this will be an issue with a majority of Spark applications in Databricks. (DT-200)

When a running job is cancelled forcefully than the application status is unpredictable as Unravel doesn't get enough time to send messages from Brace. (DT-202)

Jobs are labeled as Workflow in several places instead of Job

The number of jobs is labeled as # of YARN Apps instead of # OF Apps”.(DT-195).

Unravel Tagging feature as described here, is not supported. This affects RBAC as well.

Accessed Partitions Section is empty. (DT-175)

For spark-submit job, the Databricks’s default Spark extraJavaOptions are overwritten by Unravel's extraJavaOptions that required for monitoring. (DT-113)

Application count is incorrect on Operations page. (DT-119)

The Graphs tab is intermittently missing.(DT-209)

In multi-cluster configurations AutoActions does not differentiate between entities of each cluster and setting up a policy targets all monitored clusters. For instance, setting up a rule to target root queue causes it to be monitored on all clusters. (AA-174)

Workaround:

Uses the internal Hadoop cluster ID instead of Unravel cluster ID/name. You must obtain the internal cluster ID in order to specify a Hadoop cluster in the policy options section. It can be obtained from HDFS namenode, where it’s stored in {dfs.namenode.name.dir}/current/VERSION. (AA-150)

In case of transport message protocol synchronization error, n exceptionally rare occasion AutoAction can be triggered up to 180 seconds after the violation occurs. No data loss is expected.

“RECENT EVENTS & ALERTS” shows the events across all clusters regardless of the currently selected cluster. (AA-127, AA-151)

Application Master level metrics, such as job metrics and job counters, are not collected by EMR sensor by default and therefore can not be used in AA policies. Collection of AM metrics can be enabled manually using “am-polling” option in EMR sensor. (AA-184)

In exceptionally rare cases AutoActions can be triggered up to 180 seconds later in case of transport message protocol synchronization error but no data loss is expected.

Table size will be shown as 0 if data is in object storage. (DATAPAGE-104)

Created and Accessed Partition details are missing. (DATAPAGE-109)

Retention Table details shows number partitions as zero (0). (DATAPAGE-98)

Spaces in names are missing in HBase table name. (UIX-1736)

Exception in unravel_jcse2, unravel_td and unravel_ja.log with external MYSQL8.0. (INSTALL-233)

You can resolve by adding either of the below settings to unravel.properties.

appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

Hive queries that do not submit any DAG have empty cluster ID and queue name. (TEZLLAP-274)

For Hive-Tez, the event "TEZ DAG VERTEX USED TOO MANY REDUCER TASKS" incorrectly reports the current value for 'hive.exec.reducers.bytes.per.reducer' as 67108864 when the configured value is less than 67108864. (TEZLLAP-282)

Search behavior is not consistent behavior across all search options/boxes. (CUSTOMER-341)

The Resources' charts display is incorrect for single Unravel deployment for multiple EMR cluster. (Platform-1438)

Selecting specific clusters for Cluster Summary and Cluster Compare in Reports > Operational Insights does not work. (PLATFORM-1610)

The UI refreshes inconsistently across features. (UIX-1741, UIX-1721)

For a large number of tables, UI Details page freezes if you try to filter the tables by labels.(UIX-1874)

The Resource Usage graph does not display the entire container Data graph. (UIX-1732)