v4.7.0.0 Release notes
Software version
Release Date: 3/May/2021
See v4.7.0.0 for download information.
Software upgrade support
Only fresh installations are supported.
Sensor upgrade
Sensor upgrade is not applicable.
Certified platforms
Review your platform's compatibility matrix before you install Unravel.
Updates to Unravel's configuration properties
No updates to user-facing properties.
New features
Platforms
Unravel version 4.7.0.0 is available for the following platforms:
Cloudera Distribution of Apache Hadoop (CDH)
Cloudera Data Platform (CDP)
Databricks Azure and Databricks AWS
Amazon Elastic MapReduce (EMR)
Databricks
AutoActions is now supported for Databricks.
Data page for Databricks supports workspaces and multiple metastores.
Amazon Mapreduce (EMR)
Chargeback details are provided for each EMR cluster.
The cost breakdown is provided across EMR, EC2, and EBS volume for each configured AWS account.
Recommendations are given on the node type to save costs based on usage.
Spark SQL insights
The following new event types are added under the Analysis tab that provides insights, suggestions, and recommendations for Spark SQL queries.
Bottlenecks
SLA
UI changes
The following reports in the Cluster page are now accessed only from the Reports page:
Tuning
TopX
Queue Analysis
Scheduled Jobs report can be accessed from the Jobs page.
User report is superseded by the Top X report.
Improvements and enhancements
Application
Show the time interval when displaying applications running for a selected time interval on the graph. (CDI-88)
After Unravel restart, if the user goes to any Applications details page, Unravel UI becomes unresponsive. (CUSTOMER-1650)
Cluster
Use max instead of avg for aggregations in all graphs. (CDI-89)
Databricks
Support All Clusters in dropdown across all pages. This is only for Databricks. (CDI-158, CDI-65)
In Workspaces, you can disable tier selection while editing a workspace. (DT-890)
Add Audit tab for Databricks. (DT-827)
Support group by App tags in Chargeback. (DT-612)
New UI for Workspace Manager page. (DT-594)
HBase
Option to disable storing Table Region metrics. (HBASE-122)
Platform
Add Europe regions for EMR. (MIG-11)
Use higher strength cipher in password encryption/decryption. (REPORT-1764)
Migration
Migration script for update ES index and add attemptId and newly added structured streaming fields.
Tez
UI changes to show diagnostics message as tooltip/popup for failed task attempts. This is only for Tez. (ASP-1030)
Unsupported
Billing is not supported on EMR and on-prem platforms.
Monitoring the expiration of the SSL Certificates and Kerberos principals in Unravel multi-cluster deployments.
Migration Planning is not supported for the following regions for Azure Data Lake:
Germany Central (Sovereign)
Germany Northeast (Sovereign)
Forecasting and Migration: In a multi-cluster environment, only a single cluster can be configured at a time; hence reports are generated only for that single cluster.
Unravel does not support multi-cluster management of combined on-prem and cloud clusters.
Unravel does not support apps belonging to the same pipeline in a multi-cluster environment but is sourced from different clusters. A pipeline can only contain apps that belong to the same cluster.
Reports are not supported on Databricks and EMR.
In Jobs > Sessions, the feature of applying recommendations and then running the newly configured app is not supported.
Pig and Cascading applications are not supported.
Bug fixes
Cluster page
Resource page - Core value in the graph is not matching value in application. (DT-888)
Cluster Overview: By State/ By Status/Inefficient Events does not allow downloading the graph. (UIX-3731)
Cluster Overview: By State/ By Status/Inefficient Events does not display a check box for user selection. (UIX-3730)
Resource page - Applications listed start time does not include in the duration of the table. (CDI-152)
X-axis time range is not synchronizing on the Resource page. (CDI-128)
Jobs page graph application count does not match with the table application count. (CDI-107)
Applications
Metrics, Vcores, and memory data are missing for MR Failed and Killed apps. (ASP-1069)
Data
Data mismatch(ES vs. API Response) for /jobs/count endpoint with all supported parameter combinations for groupBy (state, applicationType, user, queue) and polling interval. (API-78)
Databricks
DBU Cost not visible in resources usage table.
Job count in the graph shows 1 but no applications listed on the Job Trends page. (CDI-130)
Exception Unrecognized field "modification_time" error in sw log. (DT-852)
Workspace Manager Page - Init script failure is not captured in the UI. (DT-598)
Click on the graph is not reflecting the details in the table. (DT-529)
Nodes graph has decimal values. (DT-460)
Impala
Impala App - Fragments and Operators details table are not scrollable intermittently. (ASP-1061)
Impala App - Left panel containing fragments and operator details not visible on Unravel UI. (ASP-1058)
Impala Table not displaying data for the selected cluster. (CDI-162)
The Impala apps details page is not opening from the Impala resource page. (ASP-1042)
Impala Option not available in the dropdown in Chargeback. (CDI-137)
Impala Resources tab is not displayed in the UI. (ASP-1025)
Impala apps are not searchable from global search. (ASP-1018)
Data IO is not displayed on the Impala app details page. (ASP-1017)
Impala: Added custom Cloudera API version support. (ASP-995)
Jobs
Jobs Page: Column sorting does not work, with cluster filter selected. (UIX-3764)
Job count in the graph shows 1 but no applications listed on the Jobs Trends page. (CDI-130)
Spark
Spark Worker not processing all yarn log paths if the first one fails. (ASP-986)
The Errors tab does not show all the errors that are displayed in the container logs. (USPARK-809)
Spark Stream batch missing jobId for few batches. (ASP-1036)
Tez
Tez application fails to load while processing dag status data. (CUSTOMER-1672)
UI
Impala: Tooltip for memory graph data points is different for both memory and query graph. (ASP-1060)
Application details pages should not show the end time for running applications. (ASP-980)
Applications are missing on the Cluster workload page. (DT-889)
Incorrect max value in slide range filter. Max, Min durations are not respective to the apps selected. (DT-882)
Pagination is not working on the Jobs page. (DT-878)
Issues in auto-logout and inactivity logout features resulting in frequent logout behaviors. (UIX-3722)
Autorefresh on the Chargeback page is not refreshing the graphs correctly. (CDI-144)
Add sub value to total available and allocated in the tooltip. (CDI-138)
Inconsistent scale used in Overview Page. (CDI-136)
Inconsistent graphs on the Overview page. (CDI-132)
JMX Monitoring UI is not loading. (CDI-129)
Cluster Resource - Multiple operations make the page unresponsive. (CDI-101)
Display full x-axis in Graphs matching the selected time range rather than when data is available. (CDI-86)
Dashboard Page - Timeline scale in the Cluster Overview page is inconsistent. (CDI-40)
Dashboard Page - vcores graph shows zero when there is no data. (CDI-39)
Use dbuCost in Chargeback pages. (DT-649)
Duplicate apps with the same cluster UID entry are found in Elastic Storage. (ASP-1044)
CDP: The application listing page does not list the Impala apps. (ASP-1076)
Total Fetch Requests per Sec metric in Kafka monitor displays much lesser value than the value displayed on Cloudera Manager. (CUSTOMER-1687)
PartitioneNotPruned is not generated for Databricks. (INSIGHTS-122)
Cluster discovery
If the metric retrieval for a host fails, then the CPU and memory capacity/usage graphs and heatmaps are not displayed.
This happens on a CDH cluster when the Cloudera Manager agent of a host does not send any heartbeats to the Cloudera Manager server. Such a host is shown as Bad Health in Cloudera Manager. (REPORT-1706)
Workaround: Ensure that the Cloudera Manager agent sends heartbeats to the Cloudera Manager on all hosts and that none of the hosts are shown as Bad Health.
The On-prem Cluster Identity may show an incorrect Spark version on CDH. The report may incorrectly show Spark 1 when Spark 2 is installed on the CDH cluster. (REPORT-1702)
The TopX report email contains a link to the Unravel TopX report, instead of showing the report content in the email as in the old reports.
Queue analysis: The log file name (
unravel_us_1.log) displayed in the error message is incorrect. The correct name of the log file isunravel_sensor.log. (REPORT-1663)
The session is not created if there is a space used in the session name. (SESS-332)
Workaround: Use underscores instead of space.
Spark Analyse session fails in the absence of event logs for the applications present in the session. (SESS-366)
Session creation from the Hive application page is not supported. (UIX-3577)
Workaround: Use the Sessions listing page to create a hive session.
The session should be sorted by create date instead of session names. (SESS-368)
Status of Application is seen as UNKNOWN in session details when the application has multiple attempts. (SESS-369)
There is a lag seen for SQL Streaming applications. (PLATFORM-2764)
If the customer uses an active directory for Kerberos and the samAccountName and principal do not match, this can cause errors when accessing HDFS.
For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)
SQL events generator generates SQL Like clause event if the query contains like pattern even in the literals. (TEZLLAP-349)
The new user interface (UI) can be accessed only from Chrome.
In the App summary page for Impala, the Query> Operator view is visible after scrolling down. (UIX-3536).
Jobs getting falsely labeled as a Tez App for Oozie Sqoop and Shell actions. (PLATFORM-2403)