Skip to main content

Home

v4.7.7.0 Release notes

Software version

Release date: 09/December/2022

See 4.7.7.0 for download information.

Software upgrade support

The following upgrade paths are supported:

  • v4.7.0.x, v4.7.1.x, v4.7.2.x, v4.7.3.x, v4.7.4.x, v4.7.5.x, v4.7.6.x → v4.7.7.0

  • v4.6.1.9 → v4.7.7.0

  • v4.6.1.8 or earlier → v4.6.1.9 → v4.7.7.0

For instructions to upgrade to Unravel v4.6.1.9, see Upgrading Unravel server.

For instructions to upgrade to Unravel v4.7.7.x, see Upgrading Unravel.

For fresh installations, see Installing Unravel.

Certified platforms

The following platforms are tested and certified in this release:

  • Google Cloud Platform (BigQuery, Dataproc)

Updates to Unravel's configuration properties

Updates to upgrading Unravel to v4.7.7.0

  • An existing license for any previous version (before 4.7.7.x) does not work with the newer version of Unravel. Therefore, before upgrading Unravel you must obtain a license file from Unravel Customer Support. For information about setting the license, see Upgrading Unravel from version 4.7.x to 4.7.7.x section in Upgrading Unravel.

  • In this release, Unravel supports the pull model, along with the push model, to get data from BigQuery projects. After the upgrade, the push model, which was used in previous releases, will be continued. However, the pull model can be implemented only with a fresh install of Unravel.

  • When you have enabled both Bigquery and Dataproc before upgrading to Unravel v4.7.7.0, then after the upgrade, you must enable Hadoop daemons (hive_worker, kafka_monitor, yarn_jc_worker, oozie_sensor) manually. see Upgrading Unravel from version 4.7.x to 4.7.7.x section in Upgrading Unravel for complete instructions.

  • For best performance, enable message ordering in GCP Cloud Pub/Sub. The Message ordering setting lets you receive the messages in the order they are published. Refer to Ordering messages for more details. After you upgrade, you cannot directly update the Message ordering setting.

    To enable the Message ordering setting after an upgrade, do the following:

    1. From the GCP console, delete the configured subscription for the respective project.

    2. Create a new subscription with the Message ordering setting on.

      Note

      The Message ordering setting can be configured only once when configuring the Pub/Sub service for the first time.

New features

  • License

    Replaced the license key with the license file for expandability and human readability. A license file contains product licensing information, the license validity, and the expiration date.

    Unravel no longer supports an automatic trial period. You must obtain a license file from Unravel Customer Support.

    For information about the Licensing documentation, see the following topics:

    Topics

    Guide

    • Setting Unravel license

    • Verifying Unravel license

    Configuration Guide

    • License: new property

    • License FAQ

    Reference Guide

    Licensing error messages and troubleshooting

    Troubleshooting Guide

    • Install Unravel on GCP Dataproc using Interactive precheck

    • Install Unravel on GCP Dataproc manually

    • Installing Unravel for GCP BigQuery

    • Manually installing Unravel for GCP BigQuery

    • Use the Interactive Precheck utility

    • Upgrading Unravel for Cloud platforms (v4.7.x to v4.7.7.0)

    Installation Guide

  • AutoActions for BigQuery

    Unravel now supports AutoActions for BigQuery. An updated user interface is provided for AutoActions in GCP with improved usability.

    For more information, refer to AutoActions > AutoActions (GCP - BigQuery, Dataproc) topic in User Guide.

  • BigQuery pull model

    Unravel now supports getting data about BigQuery from GCP with the pull model. Previously, only the push model was supported. The pull model is available only for fresh installations.

    Refer to the Installation Guide > Google Cloud platform - BigQuery installation section for more details.

  • Secure connection with Cloud SQL Auth proxy to external MySQL and PostgreSQL database

    For the Google Cloud Platform (GCP), you can enable a secure connection between Unravel server and an external MySQL or PostgreSQL database. This can be done using the Cloud SQL Auth proxy, which is a Cloud SQL connector that provides secure access to your instances without needing Authorized networks for configuring SSL. Refer to Enabling secure connection from Unravel GCP to external MySQL database with Cloud SQL Auth proxy section in the Configuration Guide.

Improvements and enhancements

  • Installation

    • Added support for the .xz archive format when configuring Unravel managed database with the --extra setup option. (INSTALL-2773)

    • When installing for BigQuery, the Interactive Precheck utility now prompts for the push and pull options. (INSTALL-2817)

  • BigQuery enhancements

    • Enhancements to the BigQuery Application Details page:

      • Added support for different job types displayed on the BigQuery details page. The supported job types are QUERY, LOAD, COPY, and EXTRACT. The new Type KPI shows the type of job run in BigQuery. (UIX-5003)

      • On the Tables tab, the tables are now categorized into Input and Output columns. (UIX-4939)

      • Added a new Child Jobs tab to display child jobs of the selected parent job. You can also view parent details in the child app. (ASP-1552)

      • Added a Close icon next to the selected stage. Click the icon to return to the App Summary page. (UIX-5045)

      For information, see the BigQuery Details page section in User Guide.

    • On-demand analysis pricing is configurable for different regions

      • You can now configure the on-demand analysis pricing for different regions, including custom prices, using the com.unraveldata.bigquery.ondemand.analysis.pricing.per.tb.usd.{region} property.

        For more information about this property, see BigQuery properties in Reference Guide.

        To configure on-demand pricing, see the Configuring on-demand pricing to process queries section in Configuration Guide.

  • Other enhancements

    • Upgraded the Node.js Javascript to the 16.18.0 version. (UIX-4461)

Unsupported

Data

  • In GCP - BigQuery, for the Data page, a count of more than 100 projects is not supported.

BigQuery pricing

  • For BigQuery pricing, Unravel only supports On-demand analysis pricing. Flat-rate analysis pricing and Storage pricing (Active and Long Term storage) is not supported.

Bug fixes

  • BigQuery

    • Incorrect column count for SelectStartEvent. (BIGQ-79)

  • Insights

    • The Elasticsearch cache issue is observed on the Insights Overview dashboard. (CDI-660)

    • After clicking the READ operator link on the Analysis tab, no action is triggered. (UIX-5067)

Known issues

Applications

  • Workflow/Job page displays empty for Analysis, Resources, Daggraph, and Errors tabs. (DT-1093)

  • Event logs and YARN logs are not loaded for some applications in Google Dataproc clusters. (ASP-1372)

AutoActions

  • When multiple AutoActions policies are created with the Overlapping ruleset and scopes, only one of the AutoAction policies is triggered. (AA-498)

  • AutoActions stop responding due to an invalid or unsupported HTTP URL or webhook. (AA-575)

BigQuery

  • On the Application details page, the original query link is missing for some cached queries due to the parallel processing of original and cached queries. (BIGQ-61)

  • Issue: Sometimes when you process a large number of BigQuery projects with the manager config bigquery integrate command, you may see the following error:

    Provider produced inconsistent result after apply

    Workaround: Wait for few minutes and re-run the command. (INSTALL-2860, INSTALL-2934)

Data page

  • If tables are created with the same name and are accessed, deleted, and re-created, and if those tables are accessed again, then their query and apps count does not match.(DATAPAGE-502)

  • For Hive metastore 3.1.0 or earlier versions, the create time of partitions is not captured if a partition is created dynamically.  Therefore, in Unravel, the “Last Day” KPIs for the partition section are not shown. (DATAPAGE-473)

Dataproc

  • Google Cloud Dataproc: Executor Logs are not loaded for Spark applications. (ASP-1371)

Email

  • Unravel node fails to send email notifications. (INSTALL-1694)

Installation

  • Issue: You can encounter a NoIndexFound exception for fresh installations of Unravel on GCP-BigQuery. (BIGQ-104)

    Workaround: Run the following CURL command on the Unravel node after the installation.

    curl -XPUT http://localhost:4171/app-19700101_07
Kerberos

  • Kerberos can only be disabled manually from the unravel.yamlfile. 

     kerberos:
      enabled: False
Reports
    Spark

    • You can see a lag for SQL Streaming applications. (PLATFORM-2764)

    Spark insights

    • For PySpark applications, the processCPUTime and processCPULoad values are not captured properly. (ASP-626)

    Tez

    • SQL events generator generates SQL Like clause event if the query contains a like pattern even in the literals. (TEZLLAP-349)

    Upgrade

    • After upgrading from v4.7.1.1 to v4.7.5.0, the Hive jobs running with the Tez application as an execution engine are not linked. (EMR-406)

    • After upgrading to v4.7.1.0, Notebooks do not work. You can configure them separately. (REPORT-1895)

    • After upgrading from v4.6.x to v4.7.1.0, the Tez application details page does not initially show DAG data. The DAG data is visible only after you refresh the page. (ASP-1126)

    UI

    • You can access the new user interface (UI) only from Chrome.

    • In the App summary page for Impala, the Query> Operator view is visible after scrolling down. (UIX-3536).

    • Issue: When you return from the application details > SQL tab> Stage page to the application details > Attempt page, the Duration, Data I/O, and Jobs Count fields are not displayed. (UIX-5048)

      Workaround: You can reload or refresh the application details > Attempt page to restore all the KPIs.

    Workflow

    • Jobs are falsely labeled as a Tez App for Oozie Sqoop and Shell actions. (PLATFORM-2403)

    Support

    For support issues, contact Unravel Support.

    In this section: