Home

v4.5.3 Release notes

Software version

Release Date: 08/09/2019

This version download is no longer available, download v4.5.3.1.

Software support

  • Upgrade from 4.5.0.x and 4.5.1.1. All that is required is a RPM upgrade.

Sensor upgrade

  • New installs: You must install the sensor.

  • Upgrades: You do not have to upgrade the sensor, but to take advantage of new features you must install it.

Certified platforms

Please review your platform's compatibility matrix below before you upgrade/install.

CDH
HDP

Updates to Unravel's configuration properties

New features

  • Applications page: Added a Running Applications tab which displays only currently running apps. (CUSTOMER-866)

  • AutoActions: Added kill action for Impala queries. (CUSTOMER-897)

  • Capacity forecasting

    • Added forecasting for memory and CPU usage. (CUSTOMER-861, REPORT-306)

  • Metrics database (EMDB): live resource metrics data is available for all Yarn apps. (REPORT-767)

  • Reports page: the following reports were added:

    • Operational Insights > Cluster KPIS: the overall health of the cluster, including apps success/failure, node health, and consumers. (CUSTOMER-871)

    • Operational Insights > Top X: Top X reports can now be configured for and emailed to specific users. (REPORT-776)

    • Migration Planning > Services and Version Compatibility: a matrix mapping your services and versions of your on-prem cluster onto a cloud platform. (REPORT-748)

  • Added support for Matomo. (CUSTOMER-260)

  • Data insights: Unravel now uses FSImage for partition and table data to populate Data Insight page which has dramatically decreased load time. (CUSTOMER-845)

Improvements

  • API: Added support API Tokens when using SAML Authentication for UI. (CUSTOMER-908)

  • AutoActions: Added support for email action to send notification to job owner with SAM account in AD. (Customer-870)

  • Data insights: Optimized Hive Table/Partitions size calculations using FSimage data. (CUSTOMER-845)

  • FSimage and small files

    • Added support for SSL connection to HiveServer2. (CUSTOMER-817)

    • Added support for AD Authentication to HiveServer2. (CUSTOMER-1004)

    • Added configuration properties to control the time and interval for triggering FSimage download.

  • Capacity forecasting

    • Unravel now stores data from these reports to allow generation of historical reports greater than 30 days. This allows storing historic time-series data for up to 2 years. (CUSTOMER-842)

    • When the source returns no data, Unravel automatically increases the start date by seven (7) days and submits a new request. (REPORT-373)

  • Impala: Added recommendation badge for Impala queries having actionable events. (CUSTOMER-865)

  • Migration planning

    • Added host type (roles) for cluster nodes (client, server, worker). (REPORT-459, CUSTOMER-778)

    • Removed Cloud Mapping per Instance tab and moved its information to top of Cloud Mapping per Host report. (REPORT-740)

    • Mapping to Google Dataproc is now supported. (REPORT-501)

    • Cloud Mapping per Host: the recommended table now displays the instance cost along with the storage cost. (REPORT-580)

    • This is now disabled by default. Set unravel.python.reporting.cloudreport.enable=true to enable it. (INSTALL-247)

  • Spark: Added recommendation/insights for long-running Spark jobs. (CUSTOMER-863)

  • Tagging: Added tagging functionality based on Database name, input tables, output tables and real user name. (CUSTOMER-1005)

  • Top X: Report now includes real user information. (CUSTOMER-996)

Bug fixes

  • Customer reported

    • Operational Insights > Impala Chargeback shows data for queries migrated from Unravel 4.3.1.x. (CUSTOMER-860)

    • AutoActions: Email for the job owner is now recovered when the AD object does not contain the owner's email. (CUSTOMER-870)

    • Password is no longer recorded in logs. (CUSTOMER-1016)

  • Impala: Killed queries are no longer classified as failed. (Impala-206)

  • Migration planning

    • Cluster Discovery and Cloud Mapping per Host reports display the cluster name. (REPORT-698)

    • Object storage pricing, which is typically tiered, is taken into account when calculating the storage costs. (REPORT-596)

  • Added support for '\' and other special characters in LDAP query as per RFC 4515. (PLATFORM-1384)

Known issues

  • AutoActions

    • Badge is not displayed for MR app killed by an AutoAction. (AA-91)

    • Auto Actions Violation Badge functionality is not working for Impala queries (Running, Killed). (AA-191)

    • Create/Update UI bug prevents user from adding individual apps by app name (only and except modes) in long-running Impala query template. (UIX-1984)

  • Cluster Optimization report is empty when one day interval is selected. (REPORT-565)

  • Data Insights page

    • Created and Accessed Partition details are missing in Overview. (DATAPAGE-109)

    • After an upgrade from 4.5.0.x to 4.5.1.x the Table KPI (total size of all tables) data is missing. (DATAPAGE-133)

      Workaround

      1. Access the database command line: /usr/local/unravel/install_bin/db_access.sh

      2. Truncate dashboard summaries table: mysql> delete from dashboard_summaries where entity_type='dkt';"

      3. Restart Unravel tw daemon: /etc/init.d/unravel_tw restart

    • When switching view by labels (All, Warm, Cold, and Hot) the page can become unresponsive. (UIX-1878)

      Workaround

      • Only solution available is to relaunch the crashed browser, and toggle ONLY ONCE, from default view ALL to any of the following labels (Hot, Warm, Cold).

  • HBase: Namespace missing in HBase table name. (UIX-1736)

  • Hive

    • Hive table details page parameters are inconsistently null. (UIX-1874)

    • Timestamp is missing for Failed MR apps. (UIX-1873)

  • Kafka is not working when clusterID has a space. (UKAFKA-69)

  • Migration Planning

    • Migration Planning Reports are not supported for MapR.

    • The following Cloud Product/Service configurations are not supported:

      • Cloud Product/Service: GCP

        Storage Type: Persistent Disks

        Region: asia

      • Cloud Product/Service: GCP

        Storage Type: Cloud Storage

        Region: asia-east1, asia, europe-north1, europe, australia, us-west1, asia-east, asia-southeast, and europe-west4

      • Cloud Product/Service: EMR and EC2

        Storage Type: S3, EBS General Purpose

        Region: Asia Pacific (Hong Kong)

    • Cloud Product/Service names displayed in the reports for Cloud Mapping per Host aren't the complete names of the Cloud Product/Service provider. (REPORT-567)

    • When Storage Type selected is Object Storage, the minimum disk space required on nodes is ignored when generating the cloud mapping. (REPORT-661)

    • The regions for Google Cloud Provider display extra regions Asia, Europe, US, Australia which should not be listed. (REPORT-663)

    • There is no ability to pick the time for the Cluster Discovery Report (you can pick only the date). The time that gets chosen (automatically) does not match the time (automatically) picked for the Chargeback reports. So the data in the charts displayed By App Type, By User and By Queue on the Cluster Discovery Report don’t match the ones on the Chargeback reports. (REPORT-684)

    • Downloaded JSON file for cloud mapping per host and cloud mapping instance display info on Workload fit which is currently not supported. (REPORT-724)

    • Cloud Mapping per Host report for Azure HDI and Azure (IaaS) cloud providers Unravel displays storage on-premises as total storage value and not the actual storage made available by Azure Managed Disks when Storage Type chosen is ‘Local Attached Storage’. (REPORT-728)

    • The “New Report” modal for Cloud Mapping per Host hangs if the user does not provide the Unravel property for cluster type (CDH/HDP) com.unraveldata.cluster.type. (REPORT-731)

    • Actual usage value utilized for figuring out cloud mapping for ‘Cost Reduction’ Strategy is based on last 6 days only of usage.

    • Migration Planning > Services and Version Compatibility: is not supported on CDH versions 6.1.1 and 6.0.1. (REPORT-889)

    • Cloud Mapping per Host fails when some regions are selected for EMR/AWS (IaaS) case. (REPORT-893)

    • Cloud Mapping per Host cost shown is incorrect when Local attached storage is selected for GCP Dataproc. (REPORT-894)

    • Unravel assumes one core on-premises is equivalent to one cloud vCPU and finds the best possible match without taking into account any performance differences. (CUSTOMER-926)

  • MR:

    • The Move to Queue option in Actions (MR APM) does not work. (PLATFORM-1195)

    • Cluster ID missing for MR apps. (PLATFORM-1434)

    • Failed MR app does have a realuser tag. (PLATFORM-1785)

  • Platform

    • appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

    • Spark worker is missing Azure libraries which causes failure when loading event log and executor logs from Azure.

    • RM Polling is delayed when Yarn kill and move actions are enabled in AutoActions. (PLATFORM-1479)

    • Setting impalad as Impala data source raises errors for invalid HTTP address. (PLATFORM-1444)

  • Queue Analysis: Graph zooming and resetting is not working properly in Edge. (UIX-158)

  • Sessions

    • Limitation in HDP cluster: While applying recommendations on a Spark session sometimes we get error like “Could not run the Spark application on the cluster: java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx”. The exact reason for this bug is yet to be ascertained.

      Workaround

      1. Open and edit spark-env.sh of the client where unravel is running.

      2. Add a line similar to export “HDP_VERSION=2.6.5.0-292". Make sure to put the proper HDP version in the line.

    • In a MapR cluster, Spark apps fail after tuning recommendations are applied. (SESS-250)

    • Hive: Newly tuned app in MapR is not shown after applying recommendations. (SESS-251)

  • Small Files does not work when sentry is enabled in the cluster. (REPORT-323)

  • UIX

    • Global search and filter by app name are not working. (UIX-1998)

    • Download CSV and XLS is not working for Tez and Spark APMs. (UIX-1999)

  • Workflow: name search is case-sensitive. (UIX-1904)

For support issues contact Unravel Support.

In this section: