Skip to main content

Home

v4.5.1.1 Release notes

Software version

Release Date: 06/03/2019

See v4.5.1.1 for specific download information.

Software support

  • Upgrade from 4.5.0.x only. All that is required is an RPM Upgrade.

Certified platforms

Please review your platform's compatibility matrix below before you upgrade/install.

CDH
HDP
Unsupported

  • ATSv2.0.

  • Small Files Report in MapR.

  • Long Running Hive Query auto action is not supported on HDP-3.0 +.

  • Migration Planning in MapR.

  • Impalad as data source for Impala queries.

  • Hive on Spark.

Unravel sensor upgrade

If you are upgrading from:

  • 4.5.0.x: you do not have to upgrade the sensor. However, it is recommended to ensure the Spark SQL app data is fully populated.

  • If you are doing a new/fresh install, you must install the new sensors.

Sensors

  • Parcel version: 1.0.4507001

  • Sensor version: 4.5.1.0rc0169

Updates to Unravel's configuration properties

  • None

New features

  • None

Improvements

  • Support added for JMX Communication over SSL for KAFKA Monitoring. (CUSTOMER-932)

  • RBAC: When enabled

    • Non-admin user can see the workflow tab. (CUSTOMER-922).

    • Applications can be filtered by the real user by setting the property com.unraveldata.rbac.default=user. (CUSTOMER-958)

Bug fixes

  • Customer reported

    • Hive and Spark can now be linked via setting com.unraveldata.com.hive.spark.enabled=true in hive-conf, or in the hive CLI or beeline session. (CDH 5.14 only.)

    • S3A file system is now supported.

    • Fixed filter to match the appropriate HDFS HA nameservice when running the Capacity Forecasting report. (CUSTOMER-993 CUSTOMER-814)

    • lmpala Queries are displayed in Data Insights page. (CUSTOMER-989 / Customer-994)

    • Workflows can be searched by workflow name. (Customer-997)

  • Top X

    • UI no longer hangs for Queries with escaped characters like '\n', '\t'. (Report-742)

    • Data displays/populates correctly. (UIX-1902)

Known issues

  • Airflow may still have scalability issues for clusters with a high number of DAGs.

  • AutoActions: Violation badge does not display for a killed application. (AA-91)

  • Cluster optimization report is empty when one day interval is selected. (REPORT-565)

  • Data insights page

    • After an upgrade from 4.5.0.9 to 4.5.1.1, the Table KPI (total size of all tables) data is missing. (DATAPAGE-109)

      Workaround

      1. Access the database command line: /usr/local/unravel/install_bin/db_access.sh

      2. Truncate dashboard summaries table: mysql> delete from dashboard_summaries where entity_type='dkt';"

      3. Restart Unravel tw daemon: /etc/init.d/unravel_tw restart

    • When switching view by labels (All, Warm, Cold,and Hot) the page can become unresponsive. (UIX-1878)

  • HBase:

    • Cluster names containing spaces are shown without the spaces, e.g., "A HBase Name" is displayed as "AHBaseName".

  • Hive

    • Hive table details page parameters are inconsistently null. (UIX-1874)

    • Timestamp is missing for Failed MR apps. (UIX-1873)

  • Impala chargeback page does not show data for queries migrated from Unravel 4.3.1.x.

  • Kafka is not working when clusterID has a space. (UKAFKA-69)

  • Migration planning

    • Migration Planning Reports are not supported for MapR.

    • The following Cloud Product/Service configurations are not supported:

      • Cloud Product/Service: GCP

        Storage Type: Persistent Disks

        Region: asia

      • Cloud Product/Service: GCP

        Storage Type: Cloud Storage

        Region: asia-east1, asia, europe-north1, europe, australia, us-west1, asia-east, asia-southeast, and europe-west4

      • Cloud Product/Service: EMR and EC2

        Storage Type: S3, EBS General Purpose

        Region: Asia Pacific (Hong Kong)

    • Cloud Mapping per Instance report displays “unsupported operand type” error message when Unravel is unable to map the host to any VM instance. (REPORT-729)

    • Unlike the Cluster Discovery report the Cloud Mapping per Host report displays the cluster ID. (REPORT-698)

    • Cloud Mapping per Host report for Azure HDI and Azure (IaaS) cloud providers Unravel displays on-prem storage as total storage value and not the actual storage made available by Azure Managed Disks when Storage Type chosen is ‘Local Attached Storage’. (REPORT-728)

    • Cloud Product/Service name displayed in the reports for Cloud Mapping per Host and Cloud Mapping per Instance are not the complete names of the Cloud Product/Service provider. (REPORT-567)

    • Downloaded JSON file for cloud mapping per host and cloud mapping instance display info on Workload fit which is currently not supported. (REPORT-724)

    • The “New Report” page for both Cloud Mapping per Instance and Cloud Mapping per Host hangs if the user does not provide the Unravel property for cluster type (CDH/HDP) property com.unraveldata.cluster.type. (REPORT-731)

    • Actual usage value utilized for figuring out cloud mapping for ‘Cost Reduction’ Strategy is based on last 6 days only of usage.

    • When Storage Type selected is Object Storage, the minimum disk space required on nodes is ignored when generating the cloud mapping. (REPORT-661)

    • There is no ability to pick the time for the Cluster Discovery Report (you can pick only the date). The time that gets chosen (automatically) does not match the time (automatically) picked for the Chargeback reports. So the data in the charts displayed By App Type, By User and By Queue on the Cluster Discovery Report don’t match the ones on the Chargeback reports. (REPORT-684)

    • The regions for Google Cloud Provider display extra regions Asia, Europe, US, Australia which should not be listed. (REPORT-663)

    • Object storage pricing is typically tiered, but is not taken into account while calculating the storage costs. (REPORT-596)

    • Unravel assumes one on-prem core is equivalent to one cloud vCPU and finds the best possible match without taking into account any performance differences. (CUSTOMER-926)

  • MR:

    • The Move to Queue option in Actions (MR APM) does not work. (PLATFORM-1195)

    • Cluster ID missing for MR apps. (PLATFORM-1434)

  • Platform

    • appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

    • Spark worker is missing Azure libraries which causes failure when loading event log and executor logs from Azure.

    • RM Polling is delayed when yarn kill and move actions are enabled in AutoActions.

    • Setting impalad as Impala data source raises errors for invalid HTTP address. (PLATFORM-1444)

    • Security

      • LDAP upgrade configs script not updating hive.server2.auth.ldap.base property to com.unraveldata.ldap.baseDN property. (INSTALL-170)

  • Queue analysis: Graph zooming and resetting is not working properly in Edge. (UIX-158)

  • Sessions

    • Limitation in HDP cluster: While applying on a Spark session sometimes we get error like “Could not run the Spark application on the cluster: java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx”. The exact reason for this bug is yet to be ascertained.

      Workaround

      1. Open and edit spark-env.sh of the client where unravel is running.

      2. Add a line similar to export “HDP_VERSION=2.6.5.0-292". Make sure to put the proper hdp version in the line.

    • In a MapR cluster, Spark application's fail after turning recommendations are applied. (SESS-250)

    • Hive: Newly tuned app in MapR is not shown after applying recommendations. (SESS-251)

  • Small files

    • Does not work when sentry is enabled in the cluster. (REPORT-323)

  • Spark

    • Load logs: Yarn logs might not get loaded if event logs are not loaded and sensor data is not available.

    • Spark SQL apps are missing data. (USPARK-310) Note: Upgrading the sensor fixes this issue.

  • Tez:

    • Vertex name is null. (TEZLLAP-227)

  • Top X report

    • Hive-on-Tez and Hive-on-Spark apps show vCore seconds and memory seconds values as zero. (TEZLLAP-249)

  • Workflow: name search is case-sensitive. (UIX-1904)

For support issues, contact Unravel Support.

In this section: