Home

v4.6.1.2 Release notes

Software version

Release Date: 06/29/2020

See v4.6.1.2 for download information

Software upgrade support

All that is required is an RPM upgrade. The following upgrade path is supported in v4.6.1.2:

  • 4.6.1.0 to 4.6.1.2

Sensor upgrade

  • A sensor upgrade is mandatory.

Certified platforms

Review your platform's compatibility matrix before you upgrade or install.

Updates to Unravel's configuration properties

Unsupported

Unravel for Databricks (Azure and AWS)

  • AutoAction

  • ADF and Airflow for interactive clusters

  • Notebooks on interactive clusters

  • Spark Program / Query Graph for Notebook and Python tasks

  • Chargeback view by custom tags

  • Cost feature for cluster type that includes spot instances. For clusters including spot instances, the predicted cost does not reflect the purchase price and outbidding events.

  • Unravel's APIs

  • Sessions

  • Role-based Access Control (RBAC)

  • Data Insights for:

    • Workload

    • Spark

  • Reports

    • Small files

    • Cluster optimization

    • Notebooks

    • Top-X

    • Forecasting

    • Migration planning

    • Queue analysis

  • Datapage

    • Size created of the table

    • Total size

    • Accessed partitions

    • Size created of the partitions

EMR

  • Datapage

    • Size created of the table.

    • Total size

    • Size created of the partitions.

  • Oozie on EMR

HDP

  • Missing table and column statistics events.

Migration Planning

Migration Planning is not supported for the following regions for Azure Data Lake:

  • US DoD East

  • US DoD Central

  • Germany Central (Sovereign)

  • Germany Northeast (Sovereign)

New features

  • Reports

    Chargeback Job Runs report is added in Operational Insights for Databricks. This report provides the chargeback Job Runs, which can be grouped by user or workspace.

  • AWS Databricks cost and instance recommendations

    Cost and instance recommendations are now supported in AWS Databricks (Standard and Premium tiers only).

  • Cost per user

    Cost per Job Run user is shown for both automated and interactive clusters.

  • Billing

    The charges incurred for Unravel monitoring can be viewed for each day in the specified period range. The billing reports display the usage and cost details that are incurred each day. Email alerts can be configured to send monthly bills for Unravel monitoring.

Improvements and enhancements

  • None

Bug fixes

  • None

Known issues

Unravel for Databricks (Azure and AWS)

  • Job Run cost is estimated based on the total number of nodes present in the cluster and irrespective of whether all the nodes are in use by the Job Run or not.

  • AWS Databricks cost supports Standard and Premium tiers only. Enterprise and Dedicated are not supported.

  • Operations dashboards do not support multi-cluster and have incorrect aggregations.

  • Operations Nodes Dashboard does not capture cluster inactivity in graphs.

  • Spark Application with the same application ID is captured as one. It is captured correctly in the Billing Service.

  • Default Databricks Spark extraJavaOptions are overwritten by Unravel for spark-submit tasks.

  • DriverOOME and ExecutorOOME events are not generated for the Databricks notebook task.

  • Recommended Azure instances available in Cluster page but not at run time.

  • Recommended Azure instances could be in Beta mode only.

  • Instance recommendation is missing when EMDB is used.

AutoActions

  • The Violation Badge functionality for AutoAction is not working for Impala queries (Running, Killed). (AA-44)

  • EMR: Hive metrics are not published in the RUNNING state. (HIVE-135)

Data Page

  • INSERT statements are not included in the App count on HDI. (DATAPAGE-256)

  • Accessed partition does not support Hive on Tez etc. It only supports Hive on MR. (DATAPAGE-250)

Platforms

  • Latency in fetching the data for MR jobs. (PLATFORM-1613)

  • API connection error while Polling impalad metrics from CM. (PLATFORM-1567)

  • conflicted ephemeral node' or 'Corrupt index found'(PLATFORM-702)

  • gc load metric sensor for MR application will not load on EMR.

Reporting

  • All files deleted except fsimage.txt (parsed fsimage) after parse fsimage is processed. (REPORT-1195)

  • Certain users and queues are missing in the Cluster Compare report. (REPORT-1342)

  • In the Cluster Compare report, the trends of the same values are highlighted instead of trends with different values.(REPORT-1479)

Spark Insights

  • For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)

  • Partition size 0 is shown in the insight message on the timings tab. (USPARK-647)

Workflow

  • The number of Apps is incorrect. (PLATFORM-2403)

For support issues, contact Unravel Support

In this section: