Skip to main content

Home

Azure Billing Integration

Integrating Azure bill with Unravel is vital to unlocking the full potential of accessing insights and reports derived from cost analysis. This integration is crucial for ensuring that the insights and reports obtained are as accurate and comprehensive as possible.

Note

If the Azure bill is not integrated with Unravel, an approximation method generates the insights. The approximate costs of DBU usage and overall expenses may vary from the values obtained directly from the Azure bill.

Integrating Azure bill with Unravel

  1. Set up Cost exports in Azure

    Before integrating with Unravel, you need to set up cost exports in your Azure environment. Configure Azure to export cost data to a designated storage account. For more details, refer to Create a Daily Export.

    Follow these steps to initiate the cost exports:

    1. Log in to the Azure portal.

    2. Access the Cost exports section.

      Cost-exports.png

    3. Create a new export by selecting the required settings and configuration options.

      Export type needs to be specified as Daily export of month to date cost.

      New-report.png

  2. Integrate Azure bill with Unravel

    Note

    Currently, only a single storage account is supported.

    There are two ways to integrate Azure bill with Unravel:

Configuring cost import with Azure storage account key

After obtaining your Azure storage account key, follow these steps:

  1. Access Azure Storage account key.

    1. Navigate to the Azure portal and locate your storage account. This is the same storage account created during the setup of cost exports in the previous section.

    2. Click the Access Keys tab in the left navigation panel to locate the storage account and its key. You can also run the following Azure CLI command to access the key:

      az storage account keys list -g MyResourceGroup -n MyStorageAccount
      Access-keys.png

  2. Configure the cost import properties.

    Run the following steps to configure the cost import:

    1. Stop Unravel

      <Unravel installation directory>/unravel/manager stop

    2. Run the following command and set the below properties using the manager tool:

      <Unravel installation directory>/unravel/manager config databricks set-cost-import <key> <name> <container> --search-directory <directory>

      Here:

      • <key> is an Azure storage account key.

      • <name> is the Azure storage account name.

      • <container> is the Azure container name where the data is stored.

      • <directory> is the directory in which the data is stored. Directory is an optional parameter.

        Note

        If the container contains any data other than the cost data, it may cause the import to fail. Therefore, specify the directory if the container contains other data as well.

      Properties

      Description

      com.unraveldata.databricks.cost.import.azure.enabled

      Determines whether the import of cost-related data from Azure Databricks into Unravel is enabled or disabled,

      com.unraveldata.databricks.cost.import.azure.account.name

      Specifies the name of the Azure account.

      com.unraveldata.databricks.cost.import.azure.account.key

      Specifies the encrypted key to the Azure account.

      com.unraveldata.databricks.cost.import.azure.container.name

      Specifies the name of the Azure storage container exports.

      com.unraveldata.databricks.cost.import.azure.search.directory

      Specifies the directory within the Azure container, which stores Azure exports.

    3. Apply the changes. If you have not stopped using Unravel services, you will be prompted to do so. Run manager stop command to stop Unravel services.

      <Unravel installation directory>/unravel/manager config apply

    4. Start Unravel

      <Unravel installation directory>/unravel/manager start
Configuring cost import with service principal and RBAC access mechanism

To configure cost import with service principal and RBAC access mechanism, perform the following steps:

  1. Stop Unravel

    <Unravel installation directory>/unravel/manager stop

  2. Run the following command and set the below properties using the manager tool:

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.enabled true
    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.account.name <account name>
    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.container.name <container name>
    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.search.directory <directory path>

    Here:

    • <account name> is the name of the Azure account.

    • <container name> is the name of the Azure container where the data is stored.

    • <directory path> is the directory in which the data is stored. Directory is an optional parameter.

      Note

      If the container contains any data other than the cost data, it may cause the import to fail. Therefore, specify the directory if the container contains other data as well.

  3. Run the following commands:

    Note

    If you have already configured Azure billing using a storage account key and are now switching to configure it using the RBAC mechanism, run the following command before executing any other commands: <Unravel installation directory>/unravel/manager config properties unset com.unraveldata.databricks.cost.import.azure.account.key

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.client.id <client id>
    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.tenant.id <azure tenent id>

  4. To configure cost import through client secret authentication, run the following command:

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.client.secret <client secret>

  5. To configure cost import through client certification authentication, run the following command:

    <Unravel installation directory>/unravel/manager config properties set com.unraveldata.databricks.cost.import.azure.client.certificate <client certificate path>

    Note

    Certificates are stored at unravel/data/certificates location.

  6. Apply the changes. If you have not stopped using Unravel services, you will be prompted to do so. Run manager stop command to stop Unravel services.

    <Unravel installation directory>/unravel/manager config apply

  7. Start Unravel

    <Unravel installation directory>/unravel/manager start
    Optional properties for the cost import

    Property/Description

    Set by user

    Unit

    Default

    com.unraveldata.databricks.cost.import.frequency.secs

    Specifies the frequency in seconds for importing cost-related data from Databricks into Unravel.

    Optional

    Integer

    3600

    com.unraveldata.databricks.cost.import.azure.endpoint

    Denotes the Azure endpoint used for importing cost-related data from Databricks into Unravel.

    Optional

    String

    https://{account_name}.blob.core.windows.net

    where {account_name} is replaced with value of com.unraveldata.databricks.cost.import.azure.container.name

    com.unraveldata.databricks.cost.import.lookback.days

    Determines the number of days to look back for importing cost-related data from Databricks into Unravel.

    Optional

    Integer

    30

    com.unraveldata.databricks.cost.import.lastrun.file

    Specifies the file used to store information about the last run of importing cost-related data from Databricks into Unravel

    Optional

    String

    $unravel_data_directory/databricks/billing/lastrun.json
    Fields obtained from Azure bill by Unravel

    Only the following fields are used for analysis from the billing API/CSV response. Some of these fields can vary based on the type of Azure plan that you have configured.

    • MeterCategory

      In MeterCategory, Unravel captures only the values of the virtual machines and Azure Databricks fields.

    • ResourceGroup

    • UsageDateTime

    • InstanceId

    • Tags

      In Tags, Unravel captures the values of the following fields:

      • ClusterId

      • ClusterName

      • Creator

      • DatabricksEnvironment

      • Vendor

      • JobId

      • Username

      • WorkspaceID

    • PreTaxCost

    • ServiceName

    • UsageQuantity

    Note

    These field names match for Pay-as-you-go accounts only. The field names could be different for enterprise accounts or different schema versions. Contact Unravel support for more information.

    In this section: