v4.5.0.5 Release notes
Software version
Release Date: 05/06/2019
See v4.5.0.5 for specific download information and Downloads for general information.
Software upgrade support
Certified platforms
Please also review the Compatibility Matrix before upgrade/install.
HDI: 3.6
EMR: 5.17, 5.18, 5.19
Supported OS and software
OS
RedHat/CentOS: 6.6-7.5
Database
MySQL: 5.6 & 5.7 (recommended)
PostgreSQL: 10.2
Database Driver
MySQL: 5.1.47
Browsers
Chrome: 70.x
Supported applications
See the Compatibility matrix for a complete listing of supported applications.
Unsupported
The following are not supported for this release:
All On-Demand Features
Sessions
Small Files
Cluster Optimization
Capacity Forecasting
Cloud Reports
Top X
Queue Analysis
File Reports
YARN logs - MR jobs
YARN aggregate logs are not captured for MR jobs. As a result, the "Logs" tab for MR jobs will be empty. Also, part of the information in the "Errors" tab is derived from these logs, so that will not be available as well.
AutoActions
HDI
Auto-scaling for cluster type Kafka.
YARN logs
The Spark jobs "Logs" tab is empty if the IndexedFile format is chosen for YARN aggregate logs. Also, part of the information in the "Errors" tab is derived from these logs, so it is not available as well.
ADLS Gen 2.
Head node HA.
Linking Spark Streaming app topic to Kafka page.
AMI/CFN/ARM Template based installation support.
No special support for cloud-native constructs e.g. IAM roles for RBAC etc.
Multi-Metastore
This is an unsupported scenario - When a cluster has more than one metastore or if an external metastore is used (as opposed to the one that comes default with the cluster).
Also, one Unravel node at a given time can only retrieve data from one metastore (it needs to be configured as described in Hive Metastore configuration documentation).
Workflows
Oozie
Airflow
Marketplace (both Azure and AWS) has older versions of Unravel, not this current 4.5.0.5 cloud release.
Improvements
Spark
Added new property, com.unraveldata.spark.shutdown.delay.ms, so the last messages are processed (allows BTrace sensor to send all the data before the spark driver exits).
Athena (Preview)
A preview of an Athena APM is available. We do not recommend you deploy this feature in a production environment.
Added support for Multiple WASB accounts for a single Unravel Deployment.
Known issues
Platform
appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)
Applications slow down and Container "not starting" errors occur when new Nodes are added to an existing EMR cluster with no Unravel bootstrap script. (EMR-4)
Multiple (ADLS Gen 1) storage accounts are not supported for a single Unravel Deployment. (HDI-33)
For HDI 3.6, Spark 2.2 or Spark 2.3 - If the format chosen for YARN aggregate logs is TFile, there is known Azure Issue that YARN aggregate logs do not get generated. This results in the "Logs" tab for Spark jobs being empty. Also, part of the information in the "Errors" tab is derived from these logs, so that will not be available as well. (USPARK-277)
Database
PostgreSQL DB is now usable across all features. However, we do not recommend it for production usage at this time.
Unravel_twcan not connect to Hive Metastore hosted on MariaDB. (CUSTOMER-716)
Data Insights Page
Data is not always displayed. (CUSTOMER-754)
Table size will be shown as 0 if data in object storage. (DATAPAGE-104)
Accessed Partition is missing. (DATAPAGE-105)
Created and Accessed Partition details are missing. (DATAPAGE-109)
Retention Table Details shows number partitions as zero (0). (DATAPAGE-98)
MR
Missing cluster ID when the app has Data IO. (PLATFORM-1430)
HBase
Table pagination is incorrect. (HBASE-95)
Spaces in names are missing in HBase table name. (UIX-1736)
The Tables RegionCount and AverageRegionSize column is blank for all tables. (UIX-1673)
Spark
The live UI for Spark applications is auto refreshed every 60 seconds, but the live data Unravel is only updated upon Job start, and Stage and Job completion. This can make the Spark Navigation tab appear static, i.e., not updated, during long-running stages.
Tez
Vertex name is null. (TEZLLAP-227)
Repeated events are showing up in the event box for the failed Tez apps. (TEZLLAP-259)
Number of events shown in annotation and UI don't match. (TEZLLAP-270)
DATA IO Info for Tez apps are intermittently missing. (TEZ-265)
java.lang.NullPointerException on ew.log. (TEZLLAP-266)
UI
Unable to filter on apps with names that contain spaces. (CUSTOMER-652)
Need more advanced search capabilities and consistent behavior across all search boxes. (CUSTOMER-341)
Selecting a different cluster does not change the list of inefficient applications on main dashboard. (PLATFORM-1418)
Single Unravel deployment for multiple EMR clusters: Resources' charts doesn't look right. (UIX-1438)
UI Refresh inconsistencies across features. (UIX-1741, UIX-1721)
Column statistics are not available against s3. (CLOUD-251)
Last update for SQL query plan is not reaching to spark worker. (USPARK-162)
Resource Usage Graph does not display the entire container Data Graph. (UIX-1732)
Hive Table is missing in Execution graph. (HIVE-78)
Reports > Operational Insights > Chargeback YARN report is not updated when a new cluster is selected. (PLATFORM-1499)
A cluster can not be selected for the following Operational Insights and Data Insights Screens. (PLATFORM-1500)
Reports > Operational Insights > Cluster Summary
Reports > Operational Insights > Cluster Compare
Reports > Operational Insights > Cluster Workload
Reports > Operational Insights > Chargeback Report
Data Insights > Overview
Data Insights > Details
For support issues, visit Unravel Support.