Home

v4.5.5.4 Release notes

Software version

Release Date: 07/16/2020

See v4.5.5.4 for download information

Software support

  • Upgrade only from 4.5.x. All that is required is an RPM upgrade.

Sensor upgrade

  • The sensor upgrade is mandatory if upgrading from Unravel versions prior to v4.5.5.2.

Certified platforms

Updates to Unravel's configuration properties

  • None

Unsupported

  • Migration Planning reports are not supported for MapR.

  • Supporting/testing MapR platforms.

New features

  • None

Improvements

  • None

Bug fixes

  • Data Page

    • Impala queries are now fetched from Elasticsearch instead of Unravel DB, thereby reducing the chances of unravel_tw failure due to out-of-memory (OOM) errors. (CUSTOMER-1449, DATAPAGE-255)

  • Impala

    • SkipQuery logic not only checks if the query is in Elasticsearch but also that the query is in one of the finished statuses (Success, Failed, Killed). (IMPALA-240)

    • The long-running Impala DDL queries move to UNKNOWN state. This issue is fixed. (CUSTOMER-1460)

  • Spark

    • The Spark applications data is now fetched from Elasticsearch instead of Unravel DB, thereby reducing multiple queries to DB. (USPARK-685)

    • Two different event types suggested the same conf parameters such as default parallelism, shuffle partitions, overriding each other. The fix removes one deprecated event. (CUSTOMER-1447)

    • The Job ID of a Spark app cannot be searched in the navigation table. The issue is fixed. (CUSTOMER-1483)

  • UI

    From the Custom Date range, when you choose a day, you can select corresponding time slot by minutes (1 min, 5 mins, 10 mins) and also by hours, instead of only the default 30 mins.

  • Upgrade

    The Unravel upgrade process can terminate prematurely when there is a large amount of data already collected in the queue_metrics table. This issue can be resolved using the available workaround.

    The following workaround should be performed only by an Unravel admin:

    Run /usr/local/unravel/install_bin/db_access.sh to get to the SQL prompt of the Unravel database. Inside the SQL prompt, execute the following:

    SHOW INDEX FROM queue_metrics;

    If you do not see "KEY `index_queue_metrics_on_time_se` (`time_se`)" in the output, execute the following:

    ALTER TABLE queue_metrics ADD INDEX `index_queue_metrics_on_time_se` (`time_se`);

    If you do not see "queue_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL" in the output, execute the following:

    ALTER TABLE queue_metrics MODIFY `queue_name` varchar(255) NOT NULL collate utf8_bin;
ALTER TABLE queue_config MODIFY `queue_name` varchar(255) NOT NULL collate utf8_bin;

    This can take some time to process based on the amount of data. It is highly recommended to run the commands such that it does not terminate when the SSH connection to Unravel host is disconnected. For example, use screen, tmux, nohup etc. (CUSTOMER-1455).

  • Workflow

    Support for Oozie 5.0+ is added. Apps are now linked appropriately to the parent workflow. (CUSTOMER-1424)

Known issues

  • AutoActions

    • Badge is not displayed for MR app killed by an AutoAction. (AA-91)

    • AutoAction's Violation Badge functionality is not working for Impala queries (Running, Killed). (AA-191)

    • Unknown transformation shutdowns unravel_aa demon when incorrect policies are set in the database. (AA-266)

    • AutoActions using duration as a violation to capture long running Hive apps are not being triggered. (AA-337)

    • Create/Update UI bug prevents users from adding individual apps by app name (only and except modes) in a long-running Impala query template. (UIX-1984)

  • Cluster optimization report is empty when

    • One day interval is selected. (REPORT-565)

  • Cluster summary report lists applications that are outside the selected time range.

  • Data insights page

    • Hive table details page parameters are inconsistently null. (DATAPAGE-189, UIX-1874)

    • Created and Accessed Partition details are missing in Overview. (DATAPAGE-109)

    • After an upgrade from 4.5.0.x to 4.5.1.x the Table KPI (total size of all tables) data is missing. (DATAPAGE-133)

      Workaround

      1. Access the database command line: /usr/local/unravel/install_bin/db_access.sh

      2. Truncate dashboard summaries table: mysql> delete from dashboard_summaries where entity_type='dkt';"

      3. Restart Unravel tw daemon: /etc/init.d/unravel_tw restart

    • When switching view by labels (All, Warm, Cold, and Hot) the page can become unresponsive. (UIX-1878)

      Workaround

      • Only solution available is to relaunch the crashed browser, and toggle ONLY ONCE, from default view ALL to any of the following labels (Hot, Warm, Cold).

  • HBase: Namespace missing in HBase table name. (UIX-1736)

  • Kafka isn't working when clusterID has a space. (UKAFKA-69)

  • Migration planning

    • Migration planning reports are not supported for MapR

    • The following Cloud product/service configurations are not supported:

      • Cloud Product/Service: Azure (IaaS), Azure (HDI)

        Storage Type: Object

        Region: Germany Northeast, South Africa North, Germany Central, US DoD East, US DoD Central, South Africa West, US Gov Virginia, US Gov Iowa.

    • Cluster Discovery Report

      • OS value is not shown for source distribution CDH 6.3. (REPORT-930)

    • Services and Version Compatibility:

      • Some of the latest versions of the components shown may be different from those present in the actual source environment. This is because the ones shown are based on the Distribution Stack's Version and not captured from the actual environment itself.

    • Cloud Mapping per Host

      • The New Report modal hangs when you don't provide the Unravel property for cluster type (CDH/HDP) com.unraveldata.cluster.type. (REPORT-731)

      • Lift and Shift: DFS Size Recommendation is different between Local Storage and Object Storage options. (REPORT-1117)

      • Unravel assumes one core on-premises is equivalent to one cloud vCPU and finds the best possible match without considering any performance differences. (REPORT-581)

      • Compatibility between selected VM type and the storage tier/class isn't verified at this time. (REPORT-940)

      • Scheduled report fails to generate a report.

        Workaround:

        Generate a report as and when needed on-demand. (REPORT-1140)

    • Cloud Mapping per Host, Workload Fit

      • Resetting/updating custom cost values doesn't work correctly. (UIX-2135, REPORT-1316, REPORT-1312).

    • Workload Fit

      • Single Cluster with auto-scaling doesn't take into consideration the minimum number of master node(s) needed. (REPORT-1018)

      • The heat map is not loaded due to Elasticsearch max size being reset. (REPORT-1375)

  • MR:

    • The Move to Queue option in Actions (MR APM) does not work. (PLATFORM-1195)

    • Cluster ID missing for MR apps. (PLATFORM-1434)

    • Timestamp is missing for Failed MR apps. (UIX-1873)

    • Not able to select last block of Histogram graph in timeline page(/UIX-2388)

  • Platform

    • While using the Open JDK version 1.8.0_252, the following error can appear:

      Error connecting to url - https://tnode69.unraveldata.com:7183/api/v15/clusters/QACDH63B1/services/hbase/roles java.lang.NullPointerException

      Workaround

      1. Open the file /usr/local/unravel/dbin/get_daemon_basics.sh. 

      2. Comment line no. 341 #ALPN_AGENT="-javaagent:${UNRAVEL_INSTALL_DIR}/dlib/agent/alpn-agent.jar"3.

      3. Add the following line:

        ALPN_AGENT=""

    • appstatus.AppInfoAccessor: Unable to update ES document. (CUSTOMER-658)

    • Spark worker is missing Azure libraries which causes failure when loading event log and executor logs from Azure.

    • RM Polling is delayed when Yarn kill and move actions are enabled in AutoActions. (PLATFORM-1479)

    • Setting impalad as Impala data source raises errors for invalid HTTP address. (PLATFORM-1444)

  • Queue Analysis: Graph zooming and resetting is not working properly in Edge. (UIX-158)

  • Small Files etl_fsimage process fails while trying to insert into the Hive Table. Workaround: use Spark to process the FSImage.

  • Sessions

    • Limitation in HDP cluster: While applying recommendations on a Spark session sometimes we get error like “Could not run the Spark application on the cluster: java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx”. The exact reason for this bug is yet to be ascertained.

      Workaround

      1. Open and edit spark-env.sh of the client where unravel is running.

      2. Add a line similar to export “HDP_VERSION=2.6.5.0-292". Make sure to put the proper HDP version in the line.

    • In a MapR cluster, Spark apps fail after tuning recommendations are applied. (SESS-250)

    • Hive: Newly tuned app in MapR is not shown after applying recommendations. (SESS-251)

  • Spark

    • Query plan and query text do not match in the Spark APM. (DT-186)

    • Some of the applications do not display both Query and Query Plan. (USPARK-511)

    • User is not directed by back to the Unravel UI when canceling print chart. (UIX-2454)

  • UIX

    • Global search and filter by app name are not working. (UIX-1998)

    • Download CSV and XLS is not working for Tez and Spark APMs. (UIX-1999)

  • Workflow

    • Name search is case-sensitive. (UIX-1904)

For support issues contact Unravel Support.

In this section: