Skip to main content

Home

v4.7.1.1 Release notes

Software version

Release Date: 17/Aug/2021

See 4.7.1.1 for download information.v4.7.1.1

Software upgrade support

The following upgrade paths are supported:

  • 4.6.1.8 or earlier → 4.6.1.9 → 4.7.1.1

  • 4.6.1.9 → 4.7.1.1

  • 4.7.0.x→ 4.7.1.1

  • 4.7.1.0 → 4.7.1.1

Sensor upgrade

  • Not applicable.

Certified platforms

The following platform is tested and certified in this release:

  • Hortonworks Data Platform (HDP)

  • Cloudera Distribution of Apache Hadoop (CDH)

  • Cloudera Data Platform (CDP)

  • Amazon Elastic MapReduce (EMR)

  • Databricks

Review your platform's compatibility matrix before you install Unravel.

Updates to Unravel's configuration properties

New features

None

Improvements and enhancements

None

Unsupported

Billing

  • Billing is not supported on EMR and on-prem platforms.

Data

  • The Data insights feature is not supported for EMR and Databricks clusters.

  • Other than Overview and Tables, the Data page is not supported for EMR and Databricks clusters.

Jobs

  • Impala jobs are not supported on the HDP platform.

Healthcheck

  • Monitoring the expiration of the SSL Certificates and Kerberos principals in Unravel multi-cluster deployments.

Migration Planning

  • Migration Planning is not supported for the following regions for Azure Data Lake:

    • Germany Central (Sovereign)

    • Germany Northeast (Sovereign)

  • Forecasting and Migration: In a multi-cluster environment, only a single cluster can be configured at a time; hence reports are generated only for that single cluster.

Multi-cluster deployment

  • Unravel does not support multi-cluster management of combined on-prem and cloud clusters.

Pipeline

  • Unravel does not support apps belonging to the same pipeline in a multi-cluster environment but is sourced from different clusters. A pipeline can only contain apps that belong to the same cluster.

Reports

  • Reports are not supported on Databricks and EMR.

Sessions

In Jobs > Sessions, the feature of applying recommendations and then running the newly configured app is not supported.

UI

  • Pig and Cascading applications are not supported.

Bug fixes

Applications, sensors, and pipeline

  • NPE in processing yarn logs when the log path is misconfigured. (ASP-1171)

  • Storing container data has issues if exit_status is populated. (ASP-1168)

  • Use generic/yarn log parser for spark container/executor log parsing. (ASP-1130)

  • DAG all vertices table KPI, in the Tez app, is shown as blank if the value is missing or 0. (ASP-1128)

Datapage

  • FSImage processing in multi-cluster fails. (DATAPAGE-504)

  • Add support for custom path for FSImage processing. (DATAPAGE-503)

  • Total Number of Tables KPI is not displayed unless apps are run on existing or new tables on a fresh start of a DataPage. (DATAPAGE-498)

  • On the Tables page, incorrect data is displayed in the Size column for tables. (DATAPAGE-496)

  • Size of Tables Created trend graph is not populated under Data Page > Overview > Tables KPIs and Trend section. (DATAPAGE-495)

  • Incorrect data is displayed in theNumber of Partitions Created trend graph under Data Page > Overview > Partitions KPIs and Trend section. (DATAPAGE-494)

  • Incorrect data displayed in Apps and Users KPIs in Tables page and Applications tab under Table Details page for a table which is dropped and recreated. (DATAPAGE-493)

  • Incorrect data displayed in Created and Owner column under Tables page. (DATAPAGE-492)

  • The data displayed for Number of Tables Accessed and Number of Users is not in sync with Number of Queries KPI data. (DATAPAGE-491)

  • FSimage feature is not getting enabled using manager config ondemand fsimage enable --automatic-fetch command. (INSTALL-1709)

  • Partitions column in Tables page displays incorrect data. (DATAPAGE-489)

  • Incorrect data displayed in the Label column for each of the tables in the downloaded data.csv file. (DATAPAGE-485)

  • The Latest Access column is not getting updated with the latest access timestamp, thereby causing wrong data to be populated in Users and Apps trend charts under Data page > Tables page. (DATAPAGE-484)

  • Incorrect data displayed for Size of Tables Created KPI/trend graph and Total Size of Tables trend graph under Data page > Tables page. (DATAPAGE-483)

  • Running FSImage leaves some KPIs in an inconsistent state until the table worker daemon is restarted. (DATAPAGE-481)

  • The Incorrect count is displayed by the Total Number of Tables KPI under Overview on the Data page. (DATAPAGE-479)

  • Number of Tables Created KPI/Trend graph does not reflect the tables data for the last day even after restarting table worker daemon where rest all KPI's reflects the data correctly. (DATAPAGE-478)

  • Apps count is not getting displayed for the expected tables in the Table page for a selected table. (DATAPAGE-469)

  • The Latest Access column does not display the date and time when the table was last accessed on the Data page > Tables page. (DATAPAGE-466)

  • The Latest Access column does not display the Date/Time when the table was last accessed in the Data page> Tables > Details table. (DATAPAGE-466)

  • Apps count is not getting displayed for the expected tables in the Table Details page for a selected table. (DATAPAGE-469)

  • Wrong database name displayed for a hive application in App Summary page (DATAPAGE-474)

  • Number of Tables Created KPI/Trend graph does not reflect the tables data for the last day even after restarting the table worker daemon. (DATAPAGE-478)

  • The wrong count is displayed for the Total Number of Tables KPI/Trend graph under the Data >Overview page. (DATAPAGE-479)

  • Running FSImage leaves some KPIs in an inconsistent state until the table worker daemon is restarted. (DATAPAGE-481)

  • Incorrect data is displayed for Size of Tables Created and Total Size of Tables KPI/trend graphs under Data page > Overview > Tables KPIs > Trends section. (DATAPAGE-483)

  • In the Data page > Tables section, the Latest Access column is not updating with the latest accessed timestamp, which causes wrong data to be populated in the Users and Apps trend charts. (DATAPAGE-484)

Databricks

  • Chargeback ES Query Modification. (DT-918)

Install

  • Health Check: No logs found: /opt/unravel/logs/pgsql/pgsql.log. (INSTALL-1756)

  • Health Check: The error, Can't list: /tmp/logs/*/logs/ is fixed. (INSTALL-1755)

  • Autoconfig: File reporting properties do not get unset when you remove the cluster configuration. (INSTALL-1733)

Spark

  • Spark Errors are not displayed completely. (ASP-1147)

  • Errors and Logs tab is empty for all previous attempts except the last attempt in the multi-attempt spark app. (ASP-849)

UI

  • The Overview page does not reflect the details on the tables created using the Northwind schema, whereas the Tables page reflects the same. (DATAPAGE-505)

  • Chargeback Download CSV shows only 15 rows. (CUSTOMER-1784)

  • In a shared Tez session, the DAG level database displayed is shared. (ASP-1146)

  • Job Runs tab shows status as a success while Spark application details page shows the status as Running. (DT-920)

  • Databricks workload page shows Impala in the drop-down list. (DT-919)

  • Load logs not loading on Unravel UI (ondemand logs). (SUPPORT-614)

  • Per page option for list (20,50,100) etc. (SUPPORT-612)

  • Label column should be renamed to State in the downloaded CSV data as this column displays the state of the table. (DATAPAGE-476)

  • Wrong database name displayed for a hive application on App Summary page. (DATAPAGE-474)

Known issues

Applications

  • Duplicate apps with the same cluster UID entry are found in Elastic Storage. (ASP-1044)

  • Event log of Spark apps fails to process if the app contains active or unfinished stages. (ASP-1175)

Data page

  • Create time of partitions does not get captured in hive meta-store if the partition is created dynamically. This limits Unravel to show Last Day KPIs for the partition section. (DATAPAGE-473)

  • Wrong data displayed for Number of Partitions Created KPI/trend graph under Partitions KPIs - Last Day section in THEData page. (DATAPAGE-473)

Email

  • Unravel node fails to send email notifications. (INSTALL-1694)

Kerberos

  • Kerberos can only be disabled manually from the unravel.yamlfile. 

     kerberos:
      enabled: False
Reports

  • Cluster discovery

    • If the metric retrieval for a host fails, then the CPU and memory capacity/usage graphs and heatmaps are not displayed.

      This happens on a CDH cluster when the Cloudera Manager agent of a host does not send any heartbeats to the Cloudera Manager server. Such a host is shown as Bad Health in Cloudera Manager. (REPORT-1706)

      Workaround: Ensure that the Cloudera Manager agent sends heartbeats to the Cloudera Manager on all hosts and that none of the hosts are shown as Bad Health.

    • The On-prem Cluster Identity may show an incorrect Spark version on CDH. The report may incorrectly show Spark 1 when Spark 2 is installed on the CDH cluster. (REPORT-1702)

  • When using PostgreSQL, the % sign is duplicated and displayed in the Workload Fit report > Map to single cluster tab. (MIG-42)

  • Cloud Mapping Per Host  report scheduled in v4.6.1.x will not work in v4.7.1.0. Users must schedule a new report. (REPORT-1886)

  • The TopX report email contains a link to the Unravel TopX report instead of showing the report content in the email as in the old reports.

  • Queue analysis: The log file name (unravel_us_1.log) displayed in the error message is incorrect. The correct name of the log file is unravel_sensor.log. (REPORT-1663)

Sensor

  • The sensor setup script fails with unrecognized arguments. (INSTALL-1667)

Spark

  • There is a lag seen for SQL Streaming applications. (PLATFORM-2764)

Security

  • If the customer uses an active directory for Kerberos and the samAccountName and principal do not match, this can cause errors when accessing HDFS. (DOC-755)

  • You cannot access Unravel UI after you upgrade from Unravel version 4.6.2.0 and enable TLS for Unravel UI. (INSTALL-1760)

    Workaround:

    Use the following workaround to get access to Unravel UI.

    1. Stop Unravel.

      <unravel_installation_directory>/unravel/manager stop

    2. Copy the key and certificates to a directory.

      cp /usr/local/unravel/data/certificates/new/hkl20043024.hk.abcd.key /usr/local/unravel/data/certificates/server_key.pem

cp /usr/local/unravel/data/certificates/new/hkl20043024.hk.abcd.pem /usr/local/unravel/data/certificates/server_cert.pem

    3. Enable the Truststore.

      <unravel_installation_directory>/unravel/manager config tls trust enable

    4. Apply changes.

      <unravel_installation_directory>/unravel/manager config apply

    5. Start Unravel.

      <unravel_installation_directory>/unravel/manager start
Spark insights

  • For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)

Tez

  • SQL events generator generates SQL Like clause event if the query contains like pattern even in the literals. (TEZLLAP-349)

  • Dag additional input and output node is not plotted for 4710 Tez applications. (ASP-1114)

Upgrade

  • Notebooks will not work after upgrading to v4.7.1.0. You can configure them separately. (REPORT-1895)

  • After you upgrade from v4.6.x to v4.7.1.0, the Tez application details page does not initially show DAG data. The DAG data is visible only after you refresh the page. (ASP-1126)

UI

  • The new user interface (UI) can be accessed only from Chrome.

  • In the App summary page for Impala, the Query> Operator view is visible after scrolling down. (UIX-3536).

Workflow

  • Jobs getting falsely labeled as a Tez App for Oozie Sqoop and Shell actions. (PLATFORM-2403)

Support

For support issues, contact Unravel Support.

In this section: