Skip to main content

Home

v4.8.1.0 Release notes

Software version

Release date: August 10, 2023

See 4.8.1.0 for download information.

Software upgrade support

Upgrade is not supported in this release.

For fresh installations, see Installing Unravel.

Certified platforms

The following platforms are tested and certified in this release:

Cloud platforms

  • Google Cloud Platform (GCP) - BigQuery

Review your platform's compatibility matrix before you install Unravel.

Updates to Unravel's configuration properties

New features

  • Installation

    • More secure ways to get BigQuery data for Unravel monitoring

      You can set up Unravel, using any of the following secure authentication methods, to receive data from BigQuery for Unravel monitoring.

      • VM-based authentication: This method is used when Unravel is hosted on a VM in the GCP compute engine service.

        In this method, a service account is created in the project in which Unravel is hosted. Instead of downloading the service account key, the service account is attached to the Unravel VM.

      • Single key-based authentication: This method is ideal for cases where Unravel is installed on a VM that is hosted outside GCP on any other cloud or on-prem environment.

        Only one service account key representing all the projects is integrated with Unravel, which is downloaded and used. This method lets you use a single key instead of multiple keys to help in key rotation.

    • Terraform is separately provided independent of the installer

      Terraform is an open-source software for infrastructure provisioning. The Terraform creates resources on the GCP account and facilitates the smooth integration of Unravel with your cloud platform. Terraform is now provided in both ways; bundled with the installer as well as separated from the installer for independent handling of resource creation and state management.

      The external Terraform is available for all the authentication (VM authentication, Single key authentication, and Multiple key authentication) methods available for integrating BigQuery projects.

      Refer to BigQuery installation in Installing Unravel.

    • Support for integration of administration projects

      The integration of administrator projects is now supported. These projects help in retrieving reservations data required for Cost 360 observability dashboards.

    • Support for integrating billing data

      You can now export and integrate BigQuery billing data for Unravel querying.

  • Capture complete cost

    Unravel can now capture the complete cost concerning BigQuery, which includes computing, storage, and services. For computing, Unravel can capture the cost for both on-demand and capacity-based (Standard, Enterprise, Enterprise Plus) pricing plans.

  • Cost optimization

    • Get 360 degree view into BigQuery spends

      You can get a detailed view of your BigQuery spending in terms of services, projects, users, and jobs.

  • Insights

    • TopK insights with the highest potential for cost optimization

      The TopK insights are provided in the following categories for cost optimization.

      • Reservations

      • Jobs

      • Scheduled Queries

      • Projects

  • Alerting

    • Set alerting based on OpenSearch to detect violations

      You can set alerting based on OpenSearch to detect violations and get alerts in your specified notification channels.

  • Jobs

    • Side-by-side job comparison

      You can compare any two jobs side-by-side to easily identify the difference between the two jobs.

Improvements and enhancements

  • App Store

    • Appstore works seamlessly for both MySQL and PostgreSQL DB over SSL/TLS. (APP-674)

  • Jobs

    • The sink query in GCP is modified to get only the jobs which has job ID. (BQ-361)

  • Properties setting

    Changed the default behavior of manager config property set command to not override by default, and instead add an explicit --force' option to override. (INSTALL-3230)

Bug fixes

None

Unsupported

Data

  • On the Data page, File Reports, Small File reports, and file size information are not supported for EMR clusters.

Platforms
AutoActions

  • Sustained Violation is not supported for Databricks AutoAction.

Amazon EMR

  • Unravel does not support the Insights Overview tab on the UI for the Amazon EMR platform.

Reports

  • All the reports, except for the TopX report, are not supported on Databricks and EMR.

Known issues

Applications

  • Workflow/Job page displays empty for Analysis, Resources, Daggraph, and Errors tabs. (DT-1093)

Data page

  • If tables are created with the same name and are accessed, deleted, and re-created, and if those tables are re-accessed, then their query and app count do not match. (DATAPAGE-502)

  • The query to fetch tableDailyKPIs is getting timed out in case of a huge table partition (27 million). (DATAPAGE-740)

  • For Hive metastore 3.1.0 or earlier versions, the creation time of partitions is not captured if a partition is created dynamically.  Therefore, the Last Day KPI for the partition section is not shown in Unravel. (DATAPAGE-473)

  • On the Data page, size data is missing for certain tables in databases, although the partition size is correctly displayed in the Partition Detail section. (DATAPAGE-695)

Databricks

  • The Job Run details page displays a duplicate entry for tasks executed during the job. (DT-1461)

  • Issue: Jobs created for PySpark application with UDF on a JobCluster fail after applying the recommendations for node downsizing. (DT-1404)

    Workaround:

    1. In your Databricks workspace, go to Configure Cluster > Advanced Options > Spark config 

    2. Add and set the following property to true for spark.driver.extraJavaOptions and spark.executor.extraJavaOptions spark configurations:

      • -Dcom.unraveldata.metrics.proctree.enable=true

      For example:

      spark.executor.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=executor,libs=spark-3.0 spark.driver.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=driver,script=StreamingProbe.btclass,libs=spark-3.0

  • Data is not displayed when you click the Optimize button corresponding to OTHERS for the Cost > Chargeback results shown in the table. (UIX-5624)

  • On the Job Runs page, the Cost slider stops working when the cost for the selected user is less than $1. (UIX-5508)

  • The job run count displayed on the Chargeback page differs from the job count shown on the Workflow page. (UIX-5581)

  • Errors and Logs data are missing in specific scenarios for a failed Spark application, such as an application failing with OutOfMemoryError. (ASP-1624)

  • On the Cost > Trends and Cost > Chargeback pages, the tooltip for the Last <number of> days field includes more days than the displayed days. (UIX-5042)

  • Clusters with an Unknown status are excluded from the dashboards often used for monitoring systems. (DT-1445)

  • When the Interactive cluster is restarted, the cluster count is increased on the Databricks Cost > Trends page. (DT-1275)

  • After navigating from the Trends and Chargeback pages with Tag filters, the No data available message is displayed on the Compute page. (DT-1094)

  • Inconsistent data is displayed for the cluster Duration and Start Time on the Compute page. (ASP-1636)

  • The DriverOOME and ExecutorOOME events are not generated for the Databricks notebook task. (DT-533)

  • When a job fails to submit a Spark application, the failed DataBricks job is missing from the Unravel UI. (ASP-1427)

  • In the Databricks view, the application is shown in a running state, even though the corresponding Spark application is marked as finished. (ASP-1436)

  • On the Workflows > Jobs page, you can view only up to 100 records or jobs. (ASI-695)

  • The Email template for the AutoAction policy contains the unformatted table. The issue occurs when the threshold value is zero (0). (ASI-688)

  • Azure Databricks jobs are randomly missing on the Unravel UI due to Azure Databricks File System (DBFS) mount issues. (PIPELINE-1626)

EMR

  • When you click the View Clusters link on the Cost-based pages and navigate to the Clusters page, the cluster numbers shown can vary. Sometimes fewer clusters are listed, and at times no clusters are shown. This is a known limitation due to differences in the definition of the time range selector for these pages. (UIX-5328)

    • Cost page

      Shows all the clusters that have accrued cost in the selected period, which may be running or terminated in the selected period, irrespective of their start date.

    • Cluster page

      Shows only those clusters that have started in the selected period.

  • Due to the known limitations, node downsizing recommendations are not suggested for the following scenarios. (EMR-519, EMR-513, EMR-424)

    • When only cluster recommendations are applied without applying Application recommendations.

    • When the workloads require high IO and partitioning.

    • When the Spark configuration spark.dynamicAllocation.enabled is true.

    • When the AWS EMR autoscaling is enabled.

    • When the workload must need parallelism (multiple CPU cores).

  • On the Clusters details page, the Insights chart is not synchronized with all other graphs displayed on the Cost tab. (EMR-618)

  • On the Cost > Trends page, users with readonly permission can create a budget. (EMR-576)

  • The workflow of multiple transient clusters (EMR) is not supported. (ASP-1424, EMR-460)

  • Unable to run Spark applications on all the master nodes after Unravel bootstrap for high availability clusters. (EMR-49)

  • Support for high availability EMR master nodes. (EMR-31)

  • For the MapReduce failed job, error details are missing on the Errors tab. (UIX-5416)

Installation

  • An exception occurs when installing Unravel version 4.7.6.0 with the Azure MySQL database (SSL Enabled). (INSTALL-2799)

Insights Overview

  • The Insights Overview tab uses UTC as the timezone, while other pages use local time. Hence, the date and time shown on the Insights Overview tab and the other pages after redirection can differ. (UIX-4176)

Kerberos

  • Kerberos can only be disabled manually from the unravel.yamlfile. 

     kerberos:
      enabled: False
Reports

  • TopX report

    • The TopX report email links to the Unravel TopX report instead of showing the report content in the email as in the old reports.

Spark

  • If the Spark job is not running for Databricks, the values for the Duration and End time fields are not updated on the Databricks Run Details page. (ASP-1616)

  • You can see a lag for SQL Streaming applications. (PLATFORM-2764)

  • On the Spark application details page, the timeline histogram is not generated correctly. (UX-632)

Spark insights

  • For PySpark applications, the processCPUTime and processCPULoad values are not captured properly. (ASP-626)

Tez

  • SQL events generator generates SQL Like clause event if the query contains a like pattern even in the literals. (TEZLLAP-349)

Upgrade

  • After upgrading from v4.7.1.1 to v4.7.5.0, the Hive jobs running with the Tez application as an execution engine are not linked. (EMR-406)

  • After upgrading to v4.7.1.0, Notebooks do not work. You can configure them separately. (REPORT-1895)

  • After upgrading from v4.6.x to v4.7.1.0, the Tez application details page does not initially show DAG data. The DAG data is visible only after you refresh the page. (ASP-1126)

Workflow

  • Jobs are falsely labeled as a Tez App for Oozie Sqoop and Shell actions. (PLATFORM-2403)

Support

For support issues, contact Unravel Support.

In this section: