Home

v4.4.1 Release notes

Software version

Release Date: 10/26/2018

For details on downloading updates see here.

Software upgrade support

Certified platforms

Please also review the Compatibility matrix before upgrade/install.

  • CDH: On-premise CDH 5.12, 5.13 & 5.14 (incl. Spark 2.3.x) with Kerberos.

  • HDP: On-premise HDP 2.6.3.0 with Kerberos & spnego enabled.

  • MapR: 6.0.0 with MapR Expansion Packs 4.1.1.

Unravel sensor upgrade

  • Yes

New features

  • Reports

    Multiple reports have been added to enable to you manage and tune your cluster and its resources. In order to use these reports, you must have OnDemand installed and use MySQL.

    Added in 4.4.0

    • You can schedule reports to run at specific times and days.

    • All report, unless noted, are archived and available to be viewed or downloaded until you delete them.

    • Cluster Optimization - generates cluster wide default recommendations to tune your cluster. Hive and MapReduce is currently supported.

    • Forecasting – uses historical disk capacity usage to extrapolate future disk needs.

    • Small Files – analyzes your directories, using your file size criteria, to help manage costly small file proliferation.

    Added in 4.4.1

    • Top Hive X: displays top Hive applications with respect to cluster usage, longest duration, and most data I/O.

    • Queue Analysis: a historical report of queues broken down apps, vcore and memory usage per queue.

    • File Reports: similar to Small Files these are canned file reports run daily using default properties set in unravel.properties. (These reports are currently not archived.)

  • Impala

    • Insights have been greatly enhanced.

    • Auto actions templates for long-running Impala queries and rogue Impala HDFS RW have been added.

  • HBase cluster

    • Support for HBase Cluster has been added.

  • Sessions

    • Provides the ability to tune/auto tune your applications.

  • PostgreSQL 10.2.1 is now the default database embedded in the RPM.

Improvements and bug fixes

Improvements
  • More than one active broker can now be displayed. (CUSTOMER-93)

  • Time picker for Kafka cluster now offers options for 1, 2, 6, and 12 hours. (CUSTOMER-94)

  • Support added for EC2 instance profiles tied to IAM roles for AWS authentication. (CUSTOMER-258)

  • Added app status "WAITING". (CUSTOMER-278)

  • Workflow instances are now configurable for past and future runs. (CUSTOMER-291)

  • Workload heat map report contains more detailed information, including ability to drill down to specific apps, users, etc. (CUSTOMER-215)

  • REST API

    • Added REST API functionality for data and application, cluster information, and chargeback. (CUSTOMER-208)

    • Added HTTPS support for API endpoints. (CUSTOMER-226)

  • AutoActions

    • Value being typed is displayed correctly when creating AutoActions. (CUSTOMER-76)

    • All metric values found in auto actions panel on application level are displayed. (CUSTOMER-207)

    • Added ability to be able to import / export AutoActions (CUSTOMER-330)

    • Support for JSON copy as an action is available from auto action list page. (CUSTOMER-344)

    • Units for auto action metrics are now shown in the UI. (Platform-667)

  • Enabled MySQL database partitioning for huge tables. (PLATfORM-306)

Bug fixes
  • Customer fixes

    • Security and vulnerability fixes. (CUSTOMER-262)

    • Hive I/O metrics shown in Application tab summary and Table tab for HDP running Hive on Tez. (CUSTOMER-298)

    • LDAP Authentication no longer ails when hive.server2.authentication.ldap.groupFilter is configured. (CUSTOMER-358)

    • Unique Stage IDs are shown in Execution tab for Spark. (CUSTOMER-236)

  • Tez

    • Tez recommendation improvements and new events. (TEZLLAP-75, TEZLLAP-111)

    • Application Timeline Server polling has been improved to ensure DAG data gets captured. (TEZLLAP-53)

    • RM diagnostic logs are now available if a Tez application or DAG fails. (TEZLLAP-128)

    • Cluster ID is now displayed for Tez applications. (TEZLLAP-31)

    • Queue name is now displayed for Tez applications. (TEZLLAP-56)

    • An application with a successful run now has correct data I/O displayed. (TEZLLAP-161)

    • A Hive on Tez application now has correct data I/O displayed. (TEZLLAP-179)

    • A Hive on Tez application now has the correct status if the corresponding Tez app has finished. (TEZLLAP-170)

    • A Tez application now has the correct status if one of the DAGs has failed. (TEZLLAP-171)

    • A link from a Tez DAG to the parent Tez application has been added. (TEZLLAP-81)

    • A NumberFormatException is no longer raised when a Tez DAG gets killed. (TEZLLAP-176)

    • Hive on Tez app-Navigation type for DAG now shows correct number of events. (UIX-1314)

  • Hive sensor

    • Hive apps now show up when there are differences in API return values in FileSinkDesc API. (SENSOR-73)

    • HiveHook support for Tez captures changed config values. (SENSOR-70)

    • Added HiveHook support for Hive >=1.2.0. (SENSOR-72)

  • Impala

    • Impala apps loading correctly. (IMPALA-139, PLATFORM-736)

  • Spark

    • Spark UI now shows information on multiple attempts. (UIX-259)

  • Kafka

    • All topic consumer groups are shown in the Kafka UI. (UIX-1296)

  • Airflow

    • Special characters in Airflow Workflow ID generation are fixed (PLATFORM-700)

    • Workflow compare graph timestamp issue fixed. (PLATFORM-703)

Known issues

  • YARN applications might be reported as RUNNING even though they're waiting to be scheduled. (PLATFORM-715)

  • Global search returns multiple results for single Hive Query ID. (UIX-1224)

  • CMP API issue picks up data ~4 days later than the earliest data point viewable. (SESS-60)

  • Auto actions are not triggering for long-running Hive apps with Tez as Execution Engine. (PLATFORM-643)

  • Delay when opening up a new workflow instance panel by clicking on Compare graph. (UIX-1357)

  • Airflow Monitoring does not work consistently. (CUSTOMER-304)

  • DFS Forecasting Report has the wrong AMS URL (REPORT-246)

Configuration properties

For support issues, contact Unravel Support.