Home

v4.5.0.5 Release notes

Software version

Release Date: 05/06/2019

See v4.5.0.5 for specific download information and Downloads for general information.

Software upgrade support
Certified platforms

Please also review the Compatibility Matrix before upgrade/install.

  • HDI: 3.6

  • EMR: 5.17, 5.18, 5.19

Supported OS and software

  • OS

    • RedHat/CentOS: 6.6-7.5

  • Database

    • MySQL: 5.6 & 5.7 (recommended)

    • PostgreSQL: 10.2

  • Database Driver

    • MySQL: 5.1.47

  • Browsers

    • Chrome: 70.x

Supported applications

See the Compatibility matrix for a complete listing of supported applications.

Unsupported

The following are not supported for this release:

  • All On-Demand Features

    • Sessions

    • Small Files

    • Cluster Optimization

    • Capacity Forecasting

    • Cloud Reports

    • Top X

    • Queue Analysis

    • File Reports

  • YARN logs - MR jobs

    • YARN aggregate logs are not captured for MR jobs. As a result, the "Logs" tab for MR jobs will be empty. Also, part of the information in the "Errors" tab is derived from these logs, so that will not be available as well.

  • AutoActions

  • HDI

    • Auto-scaling for cluster type Kafka.

    • YARN logs

      • The Spark jobs "Logs" tab is empty if the IndexedFile format is chosen for YARN aggregate logs. Also, part of the information in the "Errors" tab is derived from these logs, so it is not available as well.

    • ADLS Gen 2.

    • Head node HA.

    • Linking Spark Streaming app topic to Kafka page.

  • AMI/CFN/ARM Template based installation support.

  • No special support for cloud-native constructs e.g. IAM roles for RBAC etc.

  • Multi-Metastore

    • This is an unsupported scenario - When a cluster has more than one metastore or if an external metastore is used (as opposed to the one that comes default with the cluster).

    • Also, one Unravel node at a given time can only retrieve data from one metastore (it needs to be configured as described in Hive Metastore configuration documentation).

  • Workflows

    • Oozie

    • Airflow

  • Marketplace (both Azure and AWS) has older versions of Unravel, not this current 4.5.0.5 cloud release.

Improvements

  • Spark

    • Added new property, com.unraveldata.spark.shutdown.delay.ms, so the last messages are processed (allows BTrace sensor to send all the data before the spark driver exits).

  • Athena (Preview)

    • A preview of an Athena APM is available. We do not recommend you deploy this feature in a production environment.

  • Added support for Multiple WASB accounts for a single Unravel Deployment.

Known issues

  • Platform

    • appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

    • Applications slow down and Container "not starting" errors occur when new Nodes are added to an existing EMR cluster with no Unravel bootstrap script. (EMR-4)

    • Multiple (ADLS Gen 1) storage accounts are not supported for a single Unravel Deployment. (HDI-33)

    • For HDI 3.6, Spark 2.2 or Spark 2.3 - If the format chosen for YARN aggregate logs is TFile, there is known Azure Issue that YARN aggregate logs do not get generated. This results in the "Logs" tab for Spark jobs being empty. Also, part of the information in the "Errors" tab is derived from these logs, so that will not be available as well. (USPARK-277)

  • Database

    • PostgreSQL DB is now usable across all features. However, we do not recommend it for production usage at this time.

    • Unravel_tw can not connect to Hive Metastore hosted on MariaDB. (CUSTOMER-716)

  • Data Insights Page

    • Data is not always displayed. (CUSTOMER-754)

    • Table size will be shown as 0 if data in object storage. (DATAPAGE-104)

    • Accessed Partition is missing. (DATAPAGE-105)

    • Created and Accessed Partition details are missing. (DATAPAGE-109)

    • Retention Table Details shows number partitions as zero (0). (DATAPAGE-98)

  • MR

    • Missing cluster ID when the app has Data IO. (PLATFORM-1430)

  • HBase

    • Table pagination is incorrect. (HBASE-95)

    • Spaces in names are missing in HBase table name. (UIX-1736)

    • The Tables RegionCount and AverageRegionSize column is blank for all tables. (UIX-1673)

  • Spark

    • The live UI for Spark applications is auto refreshed every 60 seconds, but the live data Unravel is only updated upon Job start, and Stage and Job completion. This can make the Spark Navigation tab appear static, i.e., not updated, during long-running stages.

  • Tez

    • Vertex name is null. (TEZLLAP-227)

    • Repeated events are showing up in the event box for the failed Tez apps. (TEZLLAP-259)

    • Number of events shown in annotation and UI don't match. (TEZLLAP-270)

    • DATA IO Info for Tez apps are intermittently missing. (TEZ-265)

    • java.lang.NullPointerException on ew.log. (TEZLLAP-266)

  • UI

    • Unable to filter on apps with names that contain spaces. (CUSTOMER-652)

    • Need more advanced search capabilities and consistent behavior across all search boxes. (CUSTOMER-341)

    • Selecting a different cluster does not change the list of inefficient applications on main dashboard. (PLATFORM-1418)

    • Single Unravel deployment for multiple EMR clusters: Resources' charts doesn't look right. (UIX-1438)

    • UI Refresh inconsistencies across features. (UIX-1741, UIX-1721)

    • Column statistics are not available against s3. (CLOUD-251)

    • Last update for SQL query plan is not reaching to spark worker. (USPARK-162)

    • Resource Usage Graph does not display the entire container Data Graph. (UIX-1732)

    • Hive Table is missing in Execution graph. (HIVE-78)

    • Reports > Operational Insights > Chargeback YARN report is not updated when a new cluster is selected. (PLATFORM-1499)

    • A cluster can not be selected for the following Operational Insights and Data Insights Screens. (PLATFORM-1500)

      • Reports > Operational Insights > Cluster Summary

      • Reports > Operational Insights > Cluster Compare

      • Reports > Operational Insights > Cluster Workload

      • Reports > Operational Insights > Chargeback Report

      • Data Insights > Overview

      • Data Insights > Details

For support issues, visit Unravel Support.

In this section: