v4.7.9.4 Release notes
Release information
Release date | 22 February 2024 |
Software | |
Configuration properties |
Announcements
Postgres upgrade to version 15.5
The bundled Postgres database has been upgraded to version 15.5. This version supports new installations and upgrades on all platforms. If any database errors are encountered during installation or upgrade, please reach out to Unravel support for assistance.
New features
Seamless integration with Azure billing
Unravel has transitioned from using an approximation algorithm to compute costs for all Databricks entities to integrating with Azure billing APIs. Starting this release, integration with Azure billing APIs provides DBU usage, DBU cost, and VM costs that match with the cloud provider values.
New Home page with insights
Home page is introduced in this release which offers detailed insights into your cloud environment.
Easily assess your cloud spending, resource utilization, and potential savings opportunities at a glance.
With intuitive visuals and actionable data, make informed decisions to optimize your cloud resources effectively.
Explore TopX to identify top cost drivers and performance bottlenecks, and delve into optimization strategies to enhance efficiency and drive savings.
New Unravel Billing page
Unravel has introduced a new billing page from this release to support our new pricing model. Previously, Unravel employed a flat pricing model for different compute types, charging customers a fixed rate. However, recognizing the evolving landscape of compute type usage and the need for greater flexibility and accuracy in billing, Unravel has introduced a new pricing strategy aligned with market trends. Unravel now charges our customers a different pricing for each compute type.
UI enhancements in the Cost Explorer page
Experience a more intuitive navigation in the Cost Explorer page previously known as the Cost page. With renamed pages and refined table headers, navigating through cost-related insights is more intuitive than ever. Drill down into your cloud spending data and delve deeper into cost allocation and resource utilization.
The following table contains key issues addressed in the 4.7.9.4 release.
ID | Description |
|---|---|
App Store | |
APP-774 | External Elasticsearch integration not supported within the App Store environment. |
APP-775 | Databricks cost anomaly detection did not function as expected when integrated with external Elasticsearch. |
Cost | |
UIX-6310 | On the Chargeback page, when tag is not provided for any application, NULL is displayed. Upon redirection from the Optimize link for the NULL tag, the Compute page shows all applications for the selected duration instead of specifically displaying applications with no tags. |
Compute | |
DT-2094 | The cluster ID is displayed instead of the cluster name for certain clusters. |
Spark | |
DT-2141 | In the Program tab, when you click the line number, the actual line of the code is not highlighted in the Spark details App Summary page . |
DT-1404 | Jobs created for the PySpark application using User-Defined Functions on a job cluster fail after applying the recommendations for node downsizing. |
PLATFORM-2764 | There is a lag for SQL Streaming applications. |
UX-632 | The timeline histogram does not generate correctly on the Spark application details page. |
Workflows | |
PIPELINE-1626, PIPELINE-1946 | The Unravel user interface may experience issues where certain Azure Databricks jobs are missing and duplicate entries appear in Databricks workflows under specific circumstances. |
The upcoming releases will include the following key fixes to enhance user experience. It is important to note that while these issues exist, there is no immediate critical impact on using the product, and users can continue to utilize its functionality with confidence.
ID | Description |
|---|---|
Cost | |
UIX-6305 | The Others category is displayed twice in legends when the number of clusters exceeds 1000 in the Chargeback page. |
Compute | |
DT-2083 | The Total Allocated Key Performance Indicators (KPIs) for Vcore and memory are not visible in the Compute > Trends page. |
UIX-6321 | All jobs in the running status are displayed in the Finished tab under Job Runs instead of showing only the finished jobs. |
Insights | |
DT-2006 | Recommendations are provided for a failed pipeline when users utilize multiple tasks with shared job clusters, and one of the tasks fails. |
DT-2125 | The UI shows a cost discrepancy for the Executor Idle time detected insight in the Databricks version 14.2 with Photon enabled. |
Spark | |
DT-1742 | The timezone for the NodeRightSizing insight event is inconsistent in the Spark details page. |
DT-2029 | Applications in a success state may inaccurately display an associated job in a running state instead of transitioning to a failed state. |
UIX-6523 | The Sort by Write feature is currently not functioning as expected in the Spark details page. |
SaaS (Free) | |
DT-2037 | In the Databricks Standard (free) environment, there is an issue where the User Flow badge obstructs pagination. |
UI | |
UIX-6281 | The cost comparison for all the instances is not displayed on the Pipeline detail page. |
Workflows | |
DT-2104 | Sorting is incorrect when the list contains both strings starting with capital and small case letters, |
Billing
Some discrepancies may occur in cost calculations due to differences between the user time zone displayed on the Compute page and the UTC-based aggregation on the Billing page. (DT-2350)
Compute
Jobs by status graphs in the Trends tab display spark application details and not the job details. Our development is actively looking into this design limitation and efforts are underway to address this in future updates to enhance the product's capabilities. (DT-2008)
Data
If tables are created with the same name, accessed, deleted, and re-created, and if those tables are re-accessed, then their query and app count do not match. (DATAPAGE-502)
Home
Home page does not display alerts on the UI when there is missing ROI data for a single day. (DT-2509)
Hovering on Total Cost Trend on the Summary tab of the Home page may display inaccurate date information. (DT-2408)
Workflows
The current implementation has a limitation where the wrong run count is displayed for the job ID when sorting by run count in the Workflows > Jobs section. This discrepancy is currently under investigation by our development team, and active efforts are being taken to resolve this issue. (UIX-6526)
Our development team is actively investigating the following Known issues and are working towards resolving them. It is important to note that while these issues exist, there is no immediate critical impact on using the product, and users can continue to utilize its functionality with confidence.
Bug ID | Description | Workaround |
|---|---|---|
App Store | ||
APP-614 | App Store tasks fail to start with SSL enabled on the MySQL database. | |
Compute | ||
PIPELINE-1636 | Inconsistent data is displayed for the cluster Duration and Start Time on the Compute page. | NA |
Cost | ||
UIX-5624 | Data is not displayed when you click the Optimize button corresponding to OTHERS for the Cost > Chargeback results shown in the table. | NA |
DT-1094 | The No data available message is displayed on the Compute page after navigating from the Trends and Chargeback pages with Tag filters. | NA |
Datapage | ||
DATAPAGE-473 | For Hive metastore 3.1.0 or earlier versions, the creation time of partitions is not captured if a partition is created dynamically. Therefore, the Last Day KPI for the partition section is not shown in Unravel. | NA |
Insights | ||
DT-1987 | There is a mismatch in the computation of costs for fleet and spot instances in Databricks clusters. This issue arises due to the unavailability of the exact node type in the cluster info response. | NA |
Performance | ||
PIPELINE-1926 | The Insight Worker daemon is experiencing performance lag, causing delays in processing insights and data analytics tasks. | NA |
ASI-933 | In the Lag setup, the Duration is not updated for running applications. The Duration should be updated every 15 minutes. | NA |
ASI-936 | In the Lag setup, the App Time data is missing in the Timing tab of many applications. | NA |
Spark | ||
PIPELINE-1616 | If the Spark job is not running for Databricks, the values for the Duration and End time fields are not updated on the Databricks Run Details page. | NA |
DT-2012 | Incorrect details are displayed on the AppSummary > Job Run page when a user repairs a previously failed job. The displayed information may not accurately reflect the repaired job's details. | NA |
UI | ||
PIPELINE-1935 | In the Pipeline details page, when you select the data for a specific date, all instances are displayed instead of displaying only the instances within a selected date. | NA |
UIX-6321 | In the Workflow section, instead of displaying only jobs completed within the selected time frame, it currently displays jobs running within the selected duration. | NA |
UIX-6263 | The | NA |
Workflows | ||
DT-1461, PIPELINE-1939, PIPELINE-1940, DT-1093, UIX-6274, PIPELINE-1924 | The UI and data exhibit inconsistencies, including problems with job run details, issues related to multiple workflow runs and UTC timestamps , empty content in workflow job pages and issues with filter values and duration display. | NA |
App Store tasks fail to start with SSL enabled on the MySQL database. (APP-614)
Stop Unravel.
<Unravel installation directory>/unravel/manager stop
Use an editor to open
<Installation_directory>/unravel/data/conf/unravel.yamlfile.In the
unravel.yamlfile, under the database > advanced > python_flags block, enter the path to the trusted certificates. For example, if Unravel is installed at /opt/unravel, you must edit theunravel.yamlfile as follows:unravel: ...snip... database: ...snip... advanced: python_flags: ssl_ca: /opt/unravel/data/certificates/trusted_certs.pem
Use the manager utility to upload the certificates.
<Unravel installation director>/manager config tls trust add --pem /path/to/certificate
For example: /opt/unravel/manager config tls trust add --pem /path/to/certificate
Enable the Truststore.
<Unravel installation directory>/manager config tls trust enable
Apply the changes and restart Unravel.
<Unravel installation directory>/unravel/manager config apply --restart