Skip to main content


BigQuery setup guide

The following table presents information about the total number of projects and billing accounts that can be integrated with Unravel Pro (BigQuery) for monitoring in Unravel SaaS.


No of projects

Monitoring projects

100 projects

Admin project


Billing account


Prerequisites for BigQuery SaaS setup

The integration of GCP projects with BigQuery consists of the following significant aspects:

This topic provides instructions on creating resources on GCP and setting the export of billing data, which are the prerequisites for integrating Unravel.

Setting exporting of the GCP billing data for Unravel

You need the following to set the exporting of the GCP billing data for Unravel:

  • Either the Billing Accounts Costs Manager role or the Billing Account Administrator role on the target Cloud Billing account.

  • BigQuery user role for the project in which you want to export the billing data.

Run the following steps to set up the exporting of the GCP billing data for the projects monitored by Unravel.

  1. From the GCP console, choose a project to export the GCP billing data.


    Ensure this is also the same project that you want Unravel to monitor.

  2. Export the GCP billing data of the billing account you want to integrate with Unravel to the chosen project. Do the following to export the billing data:

    1. Search for the Billing Exports page and select the Cloud billing account you want to integrate with Unravel.

    2. In the Detailed Usage Cost section, click the Edit Settings button and choose the project you selected to export the GCP billing data.

    3. From the Dataset ID drop-down, select a dataset to export the billing data or create a new dataset.

    4. Confirm that the project and the dataset in which you want to export the billing data are correct, and then click Save.

    5. In the Billing export page > Detailed usage cost section, check and ensure that the Detailed usage cost is shown as Enabled and your selected project ID and dataset are displayed.

    The table creation process in the chosen dataset will take a few minutes.

  3. Note the following billing export details and keep them handy:

    • Dataset ID

    • Table ID

    • Monitoring project IDs

    • Admin project ID


      The administrator project manages the BigQuery Reservations resources and is the primary billing source for these resources. This project need not be the same project with BigQuery jobs. Google recommends creating a dedicated project for Reservations resources.

Creating GCP resources for projects monitored by Unravel

Terraform scripts are used to facilitate the smooth integration of Unravel with BigQuery projects. These scripts help you set up various components and resources.


Unravel recommends creating GCP resources for integration using the Terraform script. You can fork the repository using a GCP cloud storage bucket to store the Terraform state and push changes to the fork every time you do the integration.

The following IAM permissions are required for creating the GCP resources. Whoever executes the Terraform script for creating GCP resources for integration must have these permissions.































Follow the instructions to create GCP resources using Terraform scripts for projects that you want to be integrated with Unravel for monitoring:

  1. Use a terminal and clone the following repository:

  2. Go to unravel-terraform-scripts/saas/bigquery folder.

  3. Duplicate the example input file named input.tfvars.example. Rename it to input.tfvars.

    cp input.tfvars.example input.tfvars
  4. Configure Terraform backend. Unravel recommends configuring a central storage for your Terraform state file.

    1. Copy the file using this command:

    2. Update the Google Storage Path, to which the user who runs Terraform has access, in the file.

  5. Using a VM editor, update the following in the input.tfvars file:

    • Project IDs

      Enter the new project IDs that you had specified for Setting exporting of the GCP billing data for Unravel in the monitoring_projects_list variable. You can specify up to 100 projects.

    • Administrator project ID

      Enter your administrator project ID in the admin_project_id variable.

    • Service account

      Enter the service account that you received in the email from Unravel in the service_account variable.


      The service account is also available from the Unravel UI > Manage iconmanage1.png > BigQuery configuration, which you can copy.

  6. Navigate to the Terraform directory and run the following commands to log in to the gcloud application. A user with the essential IAM permissions can only log on to the gcloud application.

    cd unravel-terraform-scripts/saas/bigquery
    gcloud init
    gcloud auth-application-default login

    After the login, run the following commands:

    terraform init
    terraform plan --var-file=input.tfvars
    terraform apply --var-file=input.tfvars

    The GCP resources are created. You can now login to the Unravel BigQuery account using the login credentials that you have received in the email.

Integrating Unravel with BigQuery projects

Do the following to integrate BigQuery with Unravel:

  1. After your Unravel account is set, you will receive an email with the following details. Keep these handy to access your Unravel UI and integrate the BigQuery projects.

    • URL to access Unravel (BigQuery) SaaS

    • Login credentials

    • Service account

  2. Run the following steps to integrate your BigQuery projects for Unravel monitoring:

    1. Complete the prerequisites to create GCP resources and set up the exporting of billing data.

    2. Click the URL that you have received in the email and access the Unravel UI.

    3. Enter the login credentials received by email on the login page. The Unravel UI is displayed.

    4. From the top right, click manage1.png and select BigQuery configuration. The BigQuery account setting page is displayed.

    5. Specify the following details:

      • In the Monitored projects section, specify the Project IDs you have integrated using Terraform.

      • In the Administrator Projects section, specify the admin project ID.

      • In the Google Cloud Billing Export Data section, specify the Project ID, Dataset ID, and Table ID. Refer to Prerequisites.

    6. Click the Save button. The BigQuery projects are successfully integrated, and you can monitor these projects from the Unravel UI. In case any errors are shown, you must fix it and click the Save button again.

You can also refer to the following videos for BigQuery prerequisites and for integrating Unravel with BigQuery projects

  • BigQuery prerequisites

  • Integrating Unravel with BigQuery projects