Home

v4.5.0.7 Release notes

Software version

Release Date: 05/06/2019

See v4.5.0.7 for download information.

Software upgrade support
Certified platforms

Please also review the Compatibility Matrix before upgrade/install.

  • CDH: On-premise CDH 6.0, 5.16, 5.15 , 5.13

  • HDP: On-premise HDP 3.0, 2.6.5, 2.6.4

  • MapR: 6.1.0, 6.0.0, 5.2.0

Supported OS, software, and browsers

  • OS

    • RedHat/CentOS: 6.6-7.5

    • SELinux with Enforcement mode on Red Hat Enterprise Linux Server release 7.4 (Maipo)

      • Targeted Policy v.28

  • Database (PostgreSQL is bundled)

    • MySQL: 5.5, 5.6, 5.7 (recommended)

  • Database driver

    • MySQL: 5.1.47

  • Browsers

    • Chrome: 70.x

    • Internet Explorer: IE 11

Unsupported

  • ATSv2.0.

  • Small Files Report in Map:R.

  • Long Running Hive Query auto action not supported on HDP-3.0 +.

Unravel sensor upgrade

If you are upgrading from

  • 4.3.x or 4.4.x you must upgrade the Unravel Sensor.

  • 4.5.x you do not have to upgrade. However not upgrading limits some features and functionality, e.g., Unravel collects more metrics which improves the functionality of AutoActions.

  • If you doing a new/fresh install you must install the new sensors.

New Sensors

  • Parcel version: 1.0.4507001

  • Sensor version: 4.5.0.7rc0084

Updates to Unravel's configuration properties
New features

  • AutoActions

    • The policy violation messages can be delivered using SlackApp's incoming webhooks.

Improvements and bug fixes
Improvements

  • AutoActions

    • Only metrics which arrive before com.unraveldata.auto.action.metric.discard.ms are discarded and not published. (AA-30)

    • Hive-on-Spark and Hive-on-MR applications now alert in running state. (HIVE-72)

    • Added async polling of RM in jcs2. (PLATFORM-1348)

    • Fixed deduplication of metric stream in AA. (AA-37)

  • Spark

    • Made it clearer to users why Unravel does not load (BTrace) Resource Metrics until app completes.

    • Made general improvements to Containers, VCores, Memory Graphs.

    • Load Logs Improvements

      • Progress is captured in the UI while Load Logs is running.

      • Event logs are automatically loaded when not already loaded.

      • AppBadge messages under 'Load Diagnostics' explain reason for failure to load.

    • Made general improvements to Spark query table.

    • Spark BTrace Sensor Improvements

      • Split Spark SQL update from sensor into multiple messages.

      • Added new property, com.unraveldata.spark.shutdown.delay.ms, so that last messages are processed (allows BTrace sensor to send all the data before the spark driver exits).

    • Spark Enhancements

      • Spark live pipeline now updates job duration and data IO while the application is running.

      • Spark jobs and Hive query are linked for Spark-on- Hive applications.

  • Airflow

    • Airflow DAGs and Tasks processing are now scalable for large clusters.

    • Added new properties to control look-back, parsing, and whether to process logs in parallel or sequentially.

  •  Sessions

    • Improved Session's algorithm for speeding up Spark applications.

  • Hive 

    • Added support for running Hive-on-MR.

    • Added support for Hive-on-Spark.

    • AutoAction duration metric is now published for running Hive-on-MR and Hive-on-Sparks allowing alerts while running.

  • OnDemand 

    • OnDemand is compatible with RHEL 7.1.

  • Kerberos

    • Added support for custom location for Kerberos client configuration. (CUSTOMER-774)

Bug fixes

  • Customer reported

    • AutoActions for workflows now working per the condition specified in policy. (CUSTOMER-632)

    • Illegal Argument errors no longer logged in unravel_tw.log. (CUSTOMER-640)

    • Queue analysis supports queue names with quotes or special characters. (CUSTOMER-657)

    • HBase: Connecting successfully to URL and the "hbase.HBaseMetricsHandler: Error connecting to url" no longer is thrown. (CUSTOMER-659)

    • Airflow DAGs are parsed correctly. (CUSTOMER-725)

    • Exceptions no longer cause disruptions in resource manager polling. (CUSTOMER-736)

    • User can name Top X reports and more comprehensively specify date ranges. (CUSTOMER-737)

    • Data Insights page is now correctly populated and displays all data. (CUSTOMER-754)

    • All MR jobs now have correct Container, Vcores, and Memory. (CUSTOMER-764)

    • Kafka Topic's KPIs now display cumulative value across all brokers for given time. (CUSTOMER-767, CUSTOMER-884)

    • Impala queries are now displayed in UI and data is correctly populated/display as user drills down (e.g., fragments). (CUSTOMER-803, CUSTOMER-860)

    • MySQL Schema Creation (using db_schema_upgrade.sh script) no longer fails. (CUSTOMER-806)

  • Queue analysis

    • Reports are now generated consistently and correctly for every request. (REPORT-316, REPORT-378)

    • Can generate reports for specific clusters. (REPORT-370)

    • Newly added queues are polled correctly. (PLATFORM-1363)

  • Spark

    • Improved Spark Session's reliability; specifically in terms of short term memory usage. (SESS-189)

  • MR and Tez

    • UI now handles unresponsive page and Containers, vCores, Memory graphs. User is informed users when metrics are not found. (UIX-1772, UIX-1773)

Known issues

  • Special characters in passwords are not supported. (CUSTOMER-440, CUSTOMER-552)

  • Platform

    • appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

    • Spark worker is missing Azure libraries which causes failure when loading event log and executor logs from Azure.

    • Upgrade from 4.5.0.3 to 4.5.0.6 on CDH-5.16 cluster 17 services stopped working. You must switch to user hdfs for services to restart. (INSTALL-145)

    • RM Polling delayed when yarn kill and move actions are enabled in AutoActions.

    • YarnRMConnector is polling JMX from standby RM instead of active. (PLATFORM-1399)

    • Setting impalad as Impala data source raises errors for invalid http address. (PLATFORM-1444)

    • Security

      • Kerberos: kerberos.principal in unravel.properties does not take effect. (INSTALL-148)

      • LDAP upgrade configs script not updating hive.server2.auth.ldap.base property to com.unraveldata.ldap.baseDN property. (INSTALL-170)

        • KB Article: https://unraveldata.zendesk.com/hc/en-us/articles/360026752672-Users-cannot-login-via-LDAP-after-upgrade-to-Unravel-4-5-0-6-from-Unravel-4-4-or-earlier-

  • Database

    • PostgreSQL DB is now usable across all features. However we do not recommend it for production usage at this time.

    • Unravel_tw is not able to connect to Hive Metastore hosted on mariadb. (CUSTOMER-716)

  • AutoActions

    • Does not work correctly for Long Running Hive apps in MapR 6.1.0. (AA-102)

    • Violation badge does not display for a killed application. (AA-91)

  • OnDemand

    • Support needed for custom SSL cert for Ambari and Cloudera Manager in OnDemand reports. (CUSTOMER-649)

  • Forecasting

    • Throws error potentially due to SSL or bad configs. (CUSTOMER-655)

    • Is not working due to missing data in Cloudera Manager. (CUSTOMER-781)

  • Data Insights page

    • Does not always show data. (CUSTOMER-754)

    • Accessed Partition is missing in Data page. (DATAPAGE-105)

    • Created and Accessed Partition details are missing. (DATAPAGE-109)

  • Small Files report

    • Does not work when sentry is enabled in the cluster. (REPORT-323)

    • fs_image download dies not trigger after install. (REPORT-262)

  • Cluster Optimization report is empty when one day interval is selected. (REPORT-565)

  • Cloud reports - You must select an instance type (< 30 types) when choosing EMR as the Cloud Provider.

  • MR

    • The Move to Queue option in Actions (MR APM) does not work. (PLATFORM-1195)

    • Missing cluster ID when the app has Data IO. (PLATFORM-1430)

    • MR Task Logs are empty as Indexed Format is not supported. (PLATFORM-1464)

  • HBase

    • Table pagination is incorrect. (HBASE-95)

    • RegionCount and AverageRegionSize column is empty for HBase tables on on-premise cluster. (HBASE-98)

    • Spaces in names are missing in HBase table name. (UIX-1736)

  • Queue Analysis

    • May not work in case of HA enabled RM. (PLATFORM- 1423)

    • Data not updating when secondary RM is active in HA configuration. (REPORT-393)

    • Graph zooming and resetting not working properly in Edge. (UIX-158)

  • Top X report

    • Hive-on-Spark queries show zero for vcore-seconds, memory-seconds and cluster usage duration. (PLATFORM-1425)

    • Parent app badges are missing for Hive-on-Spark apps. (REPORT-535)

    • Hive-on-Tez apps show vcore seconds and memory seconds values as zero. (TEZLLAP-249)

  • Impala Chargeback

    • Graph index does not have a tool tip to display the complete text. (UIX-1834)

    • UI should display "No Data Available" message when graph is not displayed. (UIX-1832)

  • Tez

    • Vertex name is null. (TEZLLAP-227)

    • Repeated events are showing up in the event box for the failed Tez apps. (TEZLLAP-259)

    • Number of events shown in annotation and UI don't match. (TEZLLAP-270)

  • Spark (Load Logs)

    • Users isn't notified if logs are already loaded. (UIX-1785)

    • Yarn logs might not get loaded if event logs are not loaded and sensor data is not available.

    • The live UI for Spark applications is auto refreshed every 60 seconds, but the live data Unravel collects is only updated upon Job start, and Stage and Job completion. This can make the Spark Navigation tab appear static, i.e., not updated, during long running stages.

  • Workflow - Subworkflow details are not getting loaded. (UIX-1838)

For support issues, contact Unravel Support.

In this section: