v4.5.0 Release notes
Software version
Release Date: 01 /14/2019
For download location, see Downloads.
Software upgrade support
Certified platforms
Please also review the Compatibility Matrix before you upgrade/install.
CDH: On-premise CDH 6.0, 5.15, 5.14
HDP: On-premise HDP 2.6.5, 2.6.4
MapR: 6.1.0, 6.0.0, 5.2.0
Supported OS and software
OS
RedHat/CentOS: 6.6-7.5
Database (PostgreSQL is bundled)
MySQL: 5.5, 5.6, 5.7 (recommended)
Database driver
MySQL: 5.1.47
Supported browsers
Chrome: 70.x
Internet Explorer: IE 11
Unravel sensor upgrade
Sensor upgrade is only required for Spark Live View / APP Actions Feature.
Updates to Unravel's configuration properties
New features
Applications
Ability to interactively 'Kill' and 'Move' YARN applications via UI. (CUSTOMER-110)
Live view for Spark Applications/Jobs
Live update of Total, Completed, Skipped, Active, and Pending Stages for every Spark Job in a Spark application. (CUSTOMER-239)
Real-time visibility of running Spark jobs show completed stage, running stage and overall progress. (CUSTOMER-111)
Insights
New Data, operator and SQL-level insights for Impala.
New efficiency and failure events for Hive/Tez.
Cloud reports enable you in planning, migrating, and managing workloads in cloud environments (Preview*).
Added several new API end points.
Retrieve app recommendations. (CUSTOMER-396)
Get finished and total number of jobs that ran on a particular date range. (CUSTOMER-367)
Get status, error, logs, summary of individual Apps. (CUSTOMER-253)
*Preview features are in beta and subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Preview features are not subject to the support SLA of official GA features. We do not recommend you deploy Preview features in a production environment.
Improvements and bug fixes
Improvements
Auto-Actions
You can add more than 2 rulesets in auto action policy. (CUSTOMER-325)
UI shows list of triggered auto actions on mouse hover over the AA2 badge for an application. (CUSTOMER-85, CUSTOMER-275, UIX-1249)
Link to the offending application is contained in email notification. (CUSTOMER-251)
Violation email is now sent to the owner using LDAP email address field. (CUSTOMER-350)
Rules for catching named workflows that have missed their SLAs are applied are no longer indiscriminately to all running workflows. (CUSTOMER-400)
Operations
Chargeback Reports now support Impala queries. (IMPALA-118)
Top-X Report now supports Spark and Hive-On-Tez. (REPORT-241)
Sessions
Improved Spark's session recommendations for bigger containers, effective broadcast size & shuffle partition size. (SESS-128, SESS-129, SESS-130)
Spark
com.unraveldata.spark.master defines default spark master mode when not available from Sensor. (USPARK-170)
com.unraveldata.spark.eventlog.appDuration prevents event log from loading if application duration is more than configured threshold. (USPARK-168)
Reporting
OnDemand is now supported via PostgreSQL DB. (REPORT-229)
Added support to disable Small Files. (REPORT-204)
Small File Report generation is faster. (REPORT-181)
Small Files/ File Reports can run even without dfsadmin privileges. (REPORT-143)
Queue Analysis now is interactive, supports MapR disk metrics, and has UI presentation and performance improvements. (REPORT-105)
Tez
Improvements in map tasks recommendations for Hive/Tez apps. (TEZLLAP-47)
Removed recommendations to increase the number of tasks for ORDER BY reducer vertices. (TEZLLAP-35)
Improvements in hive.tez.auto.reducer.parallelism recommendations for Hive/Tez apps. (TEZLLAP-37)
Improvements in failure events for Hive/Tez apps. (TEZLLAP-80)
Added support for Hive/Tez on MapR 5.2 and 6.0. (TEZLLAP-103)
A failure event is now triggered when a Tez DAG fails. (TEZLLAP-145)
Hive queries executed as Tez apps are now connected with their Tez parent app in the UI. (TEZLLAP-199)
Hive query metrics are now published to the Auto-Actions framework. (TEZLLAP-206)
Impala
Input tables used by Impala queries are now displayed in the Data Page. (IMPALA-121)
DML statements are now captured. (IMPALA-166)
Cluster name is now available when using impalad as the data source. (IMPALA-78)
UIX
UI shows list of triggered auto actions on mouse hover over the AA2 badge. (UIX-1249)
Queue analysis
Graphs are now interactive and clicking on any of the graphs will open up the cluster view for that point in time with a list of all apps that were running at that time. (UIX-1443)
Average values for each queue metric is now displayed next to its value when selecting a point on a graph. (UIX-1493)
Average run apps, used memory, used vCores, and used disk (MaprR only) now are included as columns in the list of analyzed queues and can be sorted by these values. (UIX-1535)
All queue metric labels now have descriptions, when hovering over the metric label a pop-up window will be shown giving a complete description. (UIX-1444)
Raw queue metrics now have a retention policy to preventing possibility of the database running out of space, default is 90 days. (PLATFORM-945)
MapR disk metrics are supported, the 4th graph "Disk Usage" is also displayed when monitoring MapR clusters. (REPORT-220)
Rendering performance of graphs was greatly improved in 4.5.0.0 and users should notice much less lag when opening queue metric graphs. (UIX-1515)
Queue metric collector sensor and queue analysis reports now fully support PostgreSQL as backend storage database alongside MySQL. (REPORT-274)
HiveHook sensors
HiveHook Support for Hive 2.2.0 & Hive 2.3.0
Support for Single HiveHook class
Bug fixes
Customer fixes
Provide 'path' filter options in Small Files Report. (CUSTOMER-371)
Support for com.unraveldata.rbac.tagcmd property. (CUSTOMER-545)
For inbound HTTPS connections to Unravel UI, provide support to restrict TLS protocols. (CUSTOMER-515)
com.unraveldata.ldap.bind_pw=bigsecret needs to support encrypted passwords. (CUSTOMER-473)
Unravel using global resources to make connections to internet. (CUSTOMER-449)
Support for RHEL6 with OnDemand Framework. (CUSTOMER-435)
Spark
Spark application/job progress (CUSTOMER-475)
Spark “Load Logs” and “Application Diagnostics” (CUSTOMER-508)
Spark-shell is generating unwanted warnings (CUSTOMER-407)
Spark execution graph display issue (CUSTOMER-283)
Execution tab needs to show unique Stage IDs (CUSTOMER-236)
Kafka
Kafka Consumer Groups not displaying (CUSTOMER-366)
Kafka page not showing all partitions in UI (CUSTOMER-544)
Application Page Filters - added ability to select multiple users and multiple queues. (CUSTOMER-491)
Sorting doesn't take into account MB vs GB vs TB. (CUSTOMER-383)
Airflow Monitoring does not work consistently. (CUSTOMER-304)
Workflow compare panel only showing 24 hours worth of data. (CUSTOMER-291)
Queue analysis reports is empty and sometimes hits 1000 field limit in ES. (CUSTOMER-471)
Sessions
Recommendations generated by Application are now consistent Inside and Outside Sessions. (SESS-138)
CMP API issue picks up data ~4 days later than the earliest data point viewable. (SESS-60)
AutoActions
AutoActions are not triggering for long-running Hive apps with Tez as Execution Engine. (PLATFORM-643)
Reporting
TOP-X UI Stability Fixes (REPORT-263)
Error handling Improvements for Small Files / File Reports (REPORT-235)
Small Files Data Inconsistencies Improvements (REPORT-209)
DFS Forecasting Report has the wrong AMS URL. (REPORT-246)
Tez
Tez apps are now visible in the UI regardless of missing or incorrect ATS configuration. (TEZLLAP-50)
A Hive/Tez query is no longer associated with multiple DAGs in the Unravel UI. (TEZLLAP-198)
Fixed NullPointerException raised in
unravel_ew_1.log
. (TEZLLAP-205)Fixed NumberFormatException raised when tez.am.grouping.max-size is used instead of tez.grouping.max-size. (TEZLLAP-215)
Fixed NullPointerException raised when dagMap is not populated. (TEZLLAP-219)
Fixed NumberFormatException raised in MapR 6.0 when tez.grouping.min-size configuration is missing. (TEZLLAP-232)
Tez apps are no longer stuck in running state in MapR 6.0. (TEZLLAP-231)
Impala
HTTP connection to CM will no longer wait indefinitely. (IMPALA-139)
Short-running Impala queries are no longer missed. (IMPALA-165)
Start and end query times use the correct timezone when using impalad as the data source. (IMPALA-164)
UIX
Queue Analysis graphs are taking long time to render. (UIX-1515)
Added Report Archives Support for PostgresSQL. (REPORT-257)
Added appsRun metric on queue analysis graph. (UIX-1476)
Global search no longer returns multiple results for single Hive Query ID. (UIX-1224)
Delay when opening up a new workflow instance panel by clicking on Compare graph. (UIX-1357)
Spark
Major performance improvement in compressed Event log parsing. (USPARK-164 )
Queue analysis
MapR queue analysis report is missing Disk metrics. (REPORT-220)
Queue metric sensor is not collecting any data if PostgreSQL is used. (REPORT-274)
ES reports index limit was increased to 10000. (PLATFORM-821)
Queue Analysis report stuck at putting data to ES. (REPORT-291)
Known issues
PostgreSQL DB is now usable across all features. However, we do not recommend it for production usage at this time.
Special characters in passwords are not supported. (CUSTOMER-440, CUSTOMER-552)
fs image download will not trigger after install. (REPORT-262)
Queue analysis
Graph zooming and resetting not working properly in Edge. (UIX-1589)
May fail to collect metrics for nested queue on HDP when configured to use Fair Scheduler due to a bug in HDP. (REPORT-294)
Queue metric sensor does not work with remote clusters if RM is not accessible via HTTP or TLS with authentication is enabled. (PLATFORM-927, PLATFORM-1052)
Sessions: Auto tune fails to apply recommendations: Error: Failed to retrieve recommendations. (SESSION-137)
Cloud reports: You must select an instance type (< 30 types) when choosing EMR as the Cloud Provider.
For support issues, contact Unravel Support.