Azure bill
Integrating Azure bill with Unravel is vital to unlocking the full potential of accessing insights and reports derived from cost analysis. This integration is crucial for ensuring that the insights and reports obtained are as accurate and comprehensive as possible.
Note
If the Azure bill is not integrated with Unravel, a method of approximation is used to generate the insights. The approximate costs of DBU usage and overall expenses may vary from the values obtained directly from the Azure bill.
Integrating Azure bill with Unravel
Set up Cost exports in Azure
Before integrating with Unravel, you need to set up cost exports in your Azure environment. Configure Azure to export cost data to a designated storage account. For more details, refer to Create a Daily Export.
Follow these steps to initiate the cost exports:
Log in to the Azure portal.
Access the Cost exports section.
Create a new export by selecting the required settings and configuration options.
Export type needs to be specified as Daily export of month to date cost.
Integrate Azure bill with Unravel
There are two ways to integrate Azure bill with Unravel:
After obtaining your Azure storage account key, follow these steps:
Access Azure Storage account key.
Navigate to the Azure portal and locate your storage account. This is the same storage account created during the setup of cost exports in the previous section.
Click the Access Keys tab in the left navigation panel to locate the storage account and its key. You can also run the following Azure CLI command to access the key:
az storage account keys list -g MyResourceGroup -n MyStorageAccount
Configure the cost import properties.
Run the following steps to configure the cost import:
Stop Unravel
<Unravel installation directory>/unravel/manager stop
Run the following command and set the below properties using the manager tool:
<Unravel installation directory>/unravel/manager config databricks set-cost-import
<key><name><container>--search-directory<directory>Here:
<key>is an Azure storage account key. It will be encrypted by the command.<name>is the Azure storage account key.<container>is the Azure container name where the data is stored.<directory>is the directory in which the data is stored. Directory is an optional parameter.Note
If the container contains any data other than the cost data, it may cause the import to fail. Therefore, specify the directory if the container contains other data as well.
Properties
Description
com.unraveldata.databricks.cost.import.azure.enabled
Determines whether the import of cost-related data from Azure Databricks into Unravel is enabled or disabled,
com.unraveldata.databricks.cost.import.azure.account.name
Specifies the name of the Azure account.
com.unraveldata.databricks.cost.import.azure.account.key
Specifies the encrypted key to the Azure account.
com.unraveldata.databricks.cost.import.azure.container.name
Specifies the name of the Azure storage container exports.
com.unraveldata.databricks.cost.import.azure.search.directory
Specifies the directory within the Azure container, which stores Azure exports.
Apply the changes. If you have not stopped using Unravel services, you will be prompted to do so. Run manager stop command to stop Unravel services.
<Unravel installation directory>/unravel/manager config apply
Start Unravel
<Unravel installation directory>/unravel/manager start
To configure cost import with service principal and RBAC access mechanism, perform the following steps:
Stop Unravel
<Unravel installation directory>/unravel/manager stop
Run the following command and set the below properties using the manager tool:
<Unravel installation directory>/unravel/manager config databricks set-cost-import com.unraveldata.databricks.cost.import.azure.enabled true
<Unravel installation directory>/unravel/manager config databricks set-cost-import com.unraveldata.databricks.cost.import.azure.account.name <
account name><Unravel installation directory>/unravel/manager config databricks set-cost-import com.unraveldata.databricks.cost.import.azure.account.key <
key><Unravel installation directory>/unravel/manager config databricks set-cost-import com.unraveldata.databricks.cost.import.azure.search.directory <directory path>
Here:
<account name>is the name of the Azure account.<key>is the encrypted key to the Azure account.<directory path>is the directory in which the data is stored. Directory is an optional parameter.Note
If the container contains any data other than the cost data, it may cause the import to fail. Therefore, specify the directory if the container contains other data as well.
Run the following commands:
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.databricks.cost.import.azure.account.key
<Unravel installation directory>/unravel/manager config set com.unraveldata.databricks.cost.import.azure.client.id <
client id><Unravel installation directory>/unravel/manager config set com.unraveldata.databricks.cost.import.azure.tenant.id <
azure tenent id>To configure cost import through client secret authentication, run the following command:
<Unravel installation directory>/unravel/manager config set com.unraveldata.databricks.cost.import.azure.client.secret <
client secret>To configure cost import through client certification authentication, run the following command:
<Unravel installation directory>/unravel/manager config set com.unraveldata.databricks.cost.import.azure.client.certificate <
client certificate path>Note
Certificates are stored at unravel/data/certificates location.
Apply the changes. If you have not stopped using Unravel services, you will be prompted to do so. Run manager stop command to stop Unravel services.
<Unravel installation directory>/unravel/manager config apply
Start Unravel
<Unravel installation directory>/unravel/manager start
Property/Description | Set by user | Unit | Default |
|---|---|---|---|
com.unraveldata.databricks.cost.import.frequency.secs Specifies the frequency in seconds for importing cost-related data from Databricks into Unravel. | Optional | Integer | 3600 |
com.unraveldata.databricks.cost.import.azure.endpoint Denotes the Azure endpoint used for importing cost-related data from Databricks into Unravel. | Optional | String | https://where {account_name} is replaced with value of com.unraveldata.databricks.cost.import.azure.container.name |
com.unraveldata.databricks.cost.import.lookback.days Determines the number of days to look back for importing cost-related data from Databricks into Unravel. | Optional | Integer | 30 |
com.unraveldata.databricks.cost.import.lastrun.file Specifies the file used to store information about the last run of importing cost-related data from Databricks into Unravel | Optional | String | $unravel_data_directory/databricks/billing/lastrun.json |
Only the following fields are used for analysis from the billing API/CSV response. Some of these fields can vary based on the type of Azure plan that you have configured.
MeterCategory
In MeterCategory, Unravel captures only the values of the virtual machines and Azure Databricks fields.
ResourceGroup
UsageDateTime
InstanceId
Tags
In Tags, Unravel captures the values of the following fields:
ClusterId
ClusterName
Creator
DatabricksEnvironment
Vendor
JobId
Username
PreTaxCost
ServiceName
UsageQuantity
Note
These field names match for Pay-as-you-go accounts only. The field names could be different for enterprise accounts or different schema versions. Contact Unravel support for more information.