Skip to main content

Home

Insights Overview (Databricks)

Note

The Insights Overview tab is a preview feature, which must be enabled on the Unravel UI.

The Insights Overview tab provides a single location to summarize the insights that Unravel aggregates across your clusters. The insights are designed to monitor and reduce costs on the cloud, improve resource efficiency, and enable app acceleration.

  • Cost (Only for Databricks platform)

    In the Databricks version, only the cost and number of clusters monitored by Unravel are available, broken down into all-purpose and job clusters, with daily values shown in trends for a given time period. Besides instance type recommendations, top groups and clusters by cost are shown, as are apps caught by AutoActions for exceeding costs.

  • Resource Efficiency

    This section provides the total number of finished and failed apps and the corresponding resources, for a given time period. It also shows top users by resource, apps with insights to improve resource efficiency, the number of tables with no app access recently, and the top users by DBU.

  • App Acceleration

    This section identifies apps with insights to improve app speedup, including specific insights such as skew, inefficient joins, queries not leveraging partitions, and so on. It also shows apps caught by AutoActions for running for too long.

    The values can be shown for the last completed 1, 7, 14, or 30 days. You can also choose a custom range to view the values. Each widget has an info icon that provides a detailed description. Some have links that redirect users to other pages in Unravel for further drill down.

Enabling the Insights Overview dashboard on the UI

  1. Stop Unravel.

    Unravel installation directory>/unravel/manager stop

  2. Set the property to enable Insights Overview dashboard on the Unravel UI.

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.valuedashboard.enabled true

  3. Optionally, you can also set the threshold for idle time for a long-running cluster.

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.valuedashboard.idle.hour <no of hours>

    Note

    The default is one hour.

    For example:

    <Unravel installation directory>/unravel/manager config properties set om.unraveldata.valuedashboard.idle.hour 2

  4. Apply changes.

    <Unravel installation directory>/unravel/manager config apply

  5. Start Unravel.

    <Unravel installation directory>/unravel/manager start
Viewing the Insights Overview dashboard

The Insights Overview dashboard must be enabled for the Unravel UI using properties. After enabling it, you can click the Insights Overview (preview) tab from the menu to view the Insights Overview dashboard. The Insights Overview dashboard is displayed with the following details.

databricks-insights-overview-main.png
Inspecting details of the performance values

  1. On the Unravel UI, click Insights Overview (preview) from the menu. The App Insights dashboard is displayed with the performance values displayed in the Cost, Resource Efficiency, and App Acceleration summary sections.

  2. In any of the summary sections, in any of the displayed metrics boxes:

    • Clickinsights-overview-link.png and you will be led to the corresponding details page in Unravel.

    • Clickinsights-overview-desc.png and a pop-up with a description of the sub-section is displayed.

    • Clickinsights-overview-zero-data.png and the reason for the zero data display in the sub-section is shown.

In this section: