Skip to main content

Home

Compute (Databricks)

Note

This tab is visible only for Databricks clusters.

The Compute tab displays the list of Databricks clusters tracked by Unravel. Each cluster has a separate tab that contains information about the cluster's metadata, KPIs, configurations, trends, and Unravel's analysis. In addition, you can filter the clusters based on the status, duration, user, workspace, cluster type, and tags. The Compute tab consists of the following tabs:

  • All: Lists all the clusters and their details.

  • Finished: Lists the clusters where the jobs have completed the runs.

  • Running: Lists the clusters where the jobs are in the running status.

  • Inefficient: Lists the inefficient clusters.

The cluster list displays the following columns:

Columns

Description

Status

Status of the cluster.

Cluster Name

Name of the Databricks cluster. Click the link of the cluster name to view the details of the cluster session.

Cluster Type

Type of the Databricks cluster.

  • Interactive: Known as all-purpose clusters, which analyze data collaboratively using interactive notebooks.

  • Automated: Known as job clusters, which run fast and robust automated jobs.

  • Automated Light: Known as light clusters, which lower jobs Light Compute pricing.

User

Name of the user running the jobs on the cluster.

Insights

Displays the (insights.png) or OrangeAlerts.png icons to indicate that there are insights or recommendations available for an inefficient cluster. Click the Cluster name link to navigate to the cluster session page, where the actual insights are provided to make the cluster efficient.

Workspace

Name of the Databricks workspace.

Start time

The start time and day of the cluster.

Duration

Total time taken by the cluster to process the jobs.

Cost

An estimated cost incurred in running the apps on the cluster. This cost is calculated based on VM and DBU prices.

Actions

Click to access the Application Details page, where more details of a specific job run on the cluster are displayed.

Filtering clusters

You can filter clusters by:

  • Selecting a date and time range of the running clusters.

  • Using ID or keyword.

  • Using cluster name

  • The left sidebar lets you filter the clusters by:

    • Status: Status of the cluster, success, failed, or killed.

    • Duration: You can use the slider to set the time range.

    • Cost: You can use the slider to filter database clusters by cost (based on the dollar). If the maximum cost is less than one dollar ($1), you cannot apply the Cost filter.

    • User: Name of user, who runs the jobs on the cluster. Click in the text box for a list of valid values.

    • Workspace: Databricks workspaces. Click in the text box for a list of valid values. Click the value to select it.

    • Cluster Type: The type of cluster; Automated or Interactive. Click in the text box for a list of valid values. Click the value to select it.

    • Tags: List of created tags. Click the checkbox next to the tag and then click in the text box for the valid values.

    • Events: This filter is only available in the Inefficient tab. You can filter the inefficient apps based on the associated events.

Viewing a cluster session
  1. On the Unravel UI, navigate to Compute > All/Finished/Running/Inefficient tab.

  2. Click the link in the Cluster Name column for the cluster you want to view. The cluster session page is displayed.

    databricks-compute-cluster-session.png

    From the Cluster session page, you can get a quick reference of the following:

    • Name of the cluster

    • Type of the cluster

    • The cost incurred for the cluster

    • Status of the job run on the cluster

    • Start time of the job on the cluster

    • End time of the job on the cluster

    • Workspaces on the cluster

    • Platform of the cluster

    The following tabs are shown for the cluster session.

Cluster Detail

The Cluster detail tab shows cluster details such as the Cluster Mode, Databricks Runtime Version, Autopilot Options, Worker Type, Driver Type, and so on.

databricks-compute-cluster-cluster-detail.png
Trends

From the Trends tab, based on the type of the cluster, Interactive, Automated, or Automated Light, you can view job trends in the cluster and the trends of the resources used for all the jobs.

databricks-compute-cluster-session-trends.png

Interactive cluster

databricks-compute-cluster-session-trends-automated.png

Automated cluster

Automated-light-cluster.png

Automated Light cluster

Configurations

From this tab, you can view all the configurations set for the chosen cluster. In the upper-right corner, click the Copy icon to copy the cluster configuration, and click the Download icon to download the configuration in a JSON file format.

configurations-tab.png