v4.4.1 Release notes
Software version
Release Date: 10/26/2018
For details on downloading updates see here.
Software upgrade support
Certified platforms
Please also review the Compatibility matrix before upgrade/install.
CDH: On-premise CDH 5.12, 5.13 & 5.14 (incl. Spark 2.3.x) with Kerberos.
HDP: On-premise HDP 2.6.3.0 with Kerberos & spnego enabled.
MapR: 6.0.0 with MapR Expansion Packs 4.1.1.
Unravel sensor upgrade
Yes
New features
Reports
Multiple reports have been added to enable to you manage and tune your cluster and its resources. In order to use these reports, you must have OnDemand installed and use MySQL.
Added in 4.4.0
You can schedule reports to run at specific times and days.
All report, unless noted, are archived and available to be viewed or downloaded until you delete them.
Cluster Optimization - generates cluster wide default recommendations to tune your cluster. Hive and MapReduce is currently supported.
Forecasting – uses historical disk capacity usage to extrapolate future disk needs.
Small Files – analyzes your directories, using your file size criteria, to help manage costly small file proliferation.
Added in 4.4.1
Top Hive X: displays top Hive applications with respect to cluster usage, longest duration, and most data I/O.
Queue Analysis: a historical report of queues broken down apps, vcore and memory usage per queue.
File Reports: similar to Small Files these are canned file reports run daily using default properties set in unravel.properties. (These reports are currently not archived.)
Impala
Insights have been greatly enhanced.
Auto actions templates for long-running Impala queries and rogue Impala HDFS RW have been added.
HBase cluster
Support for HBase Cluster has been added.
Sessions
Provides the ability to tune/auto tune your applications.
PostgreSQL 10.2.1 is now the default database embedded in the RPM.
Improvements and bug fixes
Improvements
More than one active broker can now be displayed. (CUSTOMER-93)
Time picker for Kafka cluster now offers options for 1, 2, 6, and 12 hours. (CUSTOMER-94)
Support added for EC2 instance profiles tied to IAM roles for AWS authentication. (CUSTOMER-258)
Added app status "WAITING". (CUSTOMER-278)
Workflow instances are now configurable for past and future runs. (CUSTOMER-291)
Workload heat map report contains more detailed information, including ability to drill down to specific apps, users, etc. (CUSTOMER-215)
REST API
Added REST API functionality for data and application, cluster information, and chargeback. (CUSTOMER-208)
Added HTTPS support for API endpoints. (CUSTOMER-226)
AutoActions
Value being typed is displayed correctly when creating AutoActions. (CUSTOMER-76)
All metric values found in auto actions panel on application level are displayed. (CUSTOMER-207)
Added ability to be able to import / export AutoActions (CUSTOMER-330)
Support for JSON copy as an action is available from auto action list page. (CUSTOMER-344)
Units for auto action metrics are now shown in the UI. (Platform-667)
Enabled MySQL database partitioning for huge tables. (PLATfORM-306)
Bug fixes
Customer fixes
Security and vulnerability fixes. (CUSTOMER-262)
Hive I/O metrics shown in Application tab summary and Table tab for HDP running Hive on Tez. (CUSTOMER-298)
LDAP Authentication no longer ails when
hive.server2.authentication.ldap.groupFilter
is configured. (CUSTOMER-358)Unique Stage IDs are shown in Execution tab for Spark. (CUSTOMER-236)
Tez
Tez recommendation improvements and new events. (TEZLLAP-75, TEZLLAP-111)
Application Timeline Server polling has been improved to ensure DAG data gets captured. (TEZLLAP-53)
RM diagnostic logs are now available if a Tez application or DAG fails. (TEZLLAP-128)
Cluster ID is now displayed for Tez applications. (TEZLLAP-31)
Queue name is now displayed for Tez applications. (TEZLLAP-56)
An application with a successful run now has correct data I/O displayed. (TEZLLAP-161)
A Hive on Tez application now has correct data I/O displayed. (TEZLLAP-179)
A Hive on Tez application now has the correct status if the corresponding Tez app has finished. (TEZLLAP-170)
A Tez application now has the correct status if one of the DAGs has failed. (TEZLLAP-171)
A link from a Tez DAG to the parent Tez application has been added. (TEZLLAP-81)
A NumberFormatException is no longer raised when a Tez DAG gets killed. (TEZLLAP-176)
Hive on Tez app-Navigation type for DAG now shows correct number of events. (UIX-1314)
Hive sensor
Hive apps now show up when there are differences in API return values in FileSinkDesc API. (SENSOR-73)
HiveHook support for Tez captures changed config values. (SENSOR-70)
Added HiveHook support for Hive >=1.2.0. (SENSOR-72)
Impala
Impala apps loading correctly. (IMPALA-139, PLATFORM-736)
Spark
Spark UI now shows information on multiple attempts. (UIX-259)
Kafka
All topic consumer groups are shown in the Kafka UI. (UIX-1296)
Airflow
Special characters in Airflow Workflow ID generation are fixed (PLATFORM-700)
Workflow compare graph timestamp issue fixed. (PLATFORM-703)
Known issues
YARN applications might be reported as RUNNING even though they're waiting to be scheduled. (PLATFORM-715)
Global search returns multiple results for single Hive Query ID. (UIX-1224)
CMP API issue picks up data ~4 days later than the earliest data point viewable. (SESS-60)
Auto actions are not triggering for long-running Hive apps with Tez as Execution Engine. (PLATFORM-643)
Delay when opening up a new workflow instance panel by clicking on Compare graph. (UIX-1357)
Airflow Monitoring does not work consistently. (CUSTOMER-304)
DFS Forecasting Report has the wrong AMS URL (REPORT-246)
Configuration properties
See here.
For support issues, contact Unravel Support.