Home

v4.6.1.4 Release notes

Software version

Release Date: 09/07/2020

See v4.6.1.4 for download information

Software upgrade support

v4.5.3.1 tarball upgrade. The following upgrade path is supported in v4.6.1.4:

  • 4.5.3.3 to 4.6.1.4

Sensor upgrade

  • A sensor upgrade is mandatory.

Certified platforms

Review your platform's compatibility matrix before you upgrade or install.

Updates to Unravel's configuration properties

Unsupported

  • Cloud platforms (Databricks, Azure HDInsight, Amazon EMR)

  • On-premise Platforms - (Hortonworks Data Platform (HDP), MapR)

  • Migration Planning is not supported for the following regions for Azure Data Lake:

    • US DoD East

    • US DoD Central

    • Germany Central (Sovereign)

    • Germany Northeast (Sovereign)

New features

  • None

Upgrade instructions

This section provides instructions to upgrade to Unravel v4.6.1.4.

Assumptions

The following assumptions are made in the upgrade instructions:

  • Unravel is installed to /unravel-ro. You can verify the correct directory name by looking for the software directory in that location. For example: /unravel-ro/software.

  • Unravel is configured with /unravel-rw as the writable location. You can verify the correct directory name by looking for a data directory. For example: /unravel-rw/data.

Upgrading to Unravel v4.6.1.4

To install Unravel from a tar file, do the following:

  1. Change directories to /unravel-ro.

  2. Download Unravel.

    curl -v https://preview.unraveldata.com/unravel/RPM/4.6.1/unravel-4.6.1.4-CDH.tar.gz -o unravel-4.6.1.4-CDH.tar.gz -u username:password

  3. Untar unravel-4.6.1.4.

    tar zxf unravel-4.6.1.4.tar.gz -C /unravel-ro

  4. Stop Unravel.

    /unravel-rw/service/manager.sh stop

  5. Run the setup script.

    /unravel-ro/unravel/versions/4.6.1.4.377/setup --migrate-ms /unravel-rw/data

  6. (Optional) Refresh the cluster configuration.

    /unravel-rw/manager config auto

  7. Start unravel.

    /unravel-rw/manager start

  8. Check the status.

    /unravel-rw/manager report

Improvements and enhancements

  • Spark

    • streaming-sql applications type is added to display stream batches and SQL queries on the spark application details page on Unravel UI. (USPARK-727)

Bug fixes

  • Reporting

    • FSImage process fails with an error message. This issue is resolved. (REPORT-1635)

Known issues

AutoAction

  • EMR: Hive metrics are not published in the RUNNING state. (HIVE-135)

Databricks (Azure and AWS)

  • Job Run cost is estimated based on the total number of nodes present in the cluster and irrespective of whether all the nodes are in use by the Job Run or not.

  • AWS Databricks cost supports Standard and Premium tiers only. Enterprise and Dedicated are not supported.

  • Operations dashboards do not support multi-cluster and have incorrect aggregations.

  • Operations Nodes Dashboard does not capture cluster inactivity in graphs.

  • Spark Application with the same application ID is captured as one. It is captured correctly in the Billing Service.

  • Default Databricks Spark extraJavaOptions are overwritten by Unravel for spark-submit tasks.

  • DriverOOME and ExecutorOOME events are not generated for the Databricks notebook task.

  • Recommended Azure instances available in the Cluster page but not at run time.

  • Recommended Azure instances could be in Beta mode only.

  • Instance recommendation is missing when EMDB is used.

Data Page

  • INSERT statements are not included in the App count on HDI. (DATAPAGE-256)

  • The accessed partition does not support Hive on Tez etc. It only supports Hive on MR. (DATAPAGE-250)

  • Datapage does not load the total table and partition size KPI's (DATAPAGE-379)

Installation

  • After a clean install, the manager config  failed to update Kerberos details in unravel.properties, even after passing the Kerberos parameter. (INSTALL-746)

    Workaround: Manually add the Kerberos configurations.

Platforms

  • gc load metric sensor for MR application will not load on EMR.

Reporting

  • Certain users and queues are missing in the Cluster Compare report. (REPORT-1342)

  • In the Cluster Compare report, the trends of the same values are highlighted instead of trends with different values. (REPORT-1479)

Spark

  • There is a lag seen while streaming Spark SQL applications. (PLATFORM-2764)

Spark Insights

  • For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)

  • Partition size 0 is shown in the insight message on the timings tab. (USPARK-647)

Workflow

  • The number of Apps is incorrect. (PLATFORM-2403)

  • Spark apps running in Oozie workflow is not visible under Oozie workflow in Unravel UI. (PLATFORM-2746)

Sessions

  • The create sessions and auto tune sessions does not appear in the events panel of SQL spark application. (SESS-330)

  • The Session apply feature does not function in a kerberized environment. However, both Session creation and Session analyze features work as expected. (SESS-329)

    Workaround: Pick the recommendations from the session and apply them to the application manually.

UI

  • For streaming applications, the global search with application ID does not work. (UIX-3312)

For support issues, contact Unravel Support

In this section: