Skip to main content

Home

Prerequisites for Unravel Pro BigQuery

After you upgrade to or sign up for Unravel Pro (BigQuery), ensure to complete the prerequisites required for integrating GCP projects. The integration of GCP projects with BigQuery consists of the following significant aspects:

This topic provides instructions on creating resources on GCP and setting the export of billing data, which are the prerequisites for integrating Unravel after upgrading to Unravel Pro from a Standard (Free) account.

Setting exporting of the GCP billing data for Unravel

When you completed the prerequisites for Standard (Free) account, you have already done the setup for exporting of GCP billing data. If not done already, refer to the Setting export of the GCP billing data section to complete the setting. You can now monitor up to 100 projects, and the configurations done for the Unravel Standard (Free) account will remain unchanged in Unravel Pro.

Creating GCP resources for projects monitored by Unravel

Terraform scripts are used to facilitate the smooth integration of Unravel with BigQuery projects. These scripts help you set up various components and resources.

Note

Unravel recommends creating GCP resources for integration using the Terraform script. You can fork the repository using a GCP cloud storage bucket to store the Terraform state and push changes to the fork every time you do the integration.

The following IAM permissions are required for creating the GCP resources. Whoever executes the Terraform script for creating GCP resources for integration must have these permissions.

iam.roles.create

iam.serviceAccountKeys.create

logging.sinks.create

pubsub.subscriptions.create

resourcemanager.projects.get

iam.roles.delete

iam.serviceAccountKeys.delete

logging.sinks.delete

pubsub.subscriptions.delete

resourcemanager.projects.getIamPolicy

iam.roles.get

iam.serviceAccountKeys.get

logging.sinks.get

pubsub.subscriptions.get

resourcemanager.projects.setIamPolicy

iam.roles.undelete

iam.serviceAccounts.create

pubsub.topics.attachSubscription

iam.roles.update

iam.serviceAccounts.delete

pubsub.topics.create

iam.serviceAccounts.get

pubsub.topics.delete

iam.serviceAccounts.getIamPolicy

pubsub.topics.get

iam.serviceAccounts.getIamPolicy

pubsub.topics.get

iam.serviceAccounts.setIamPolicy

pubsub.topics.getIamPolicy

pubsub.topics.setIamPolicy

Follow the instructions to create GCP resources using Terraform scripts for projects that you want to be integrated with Unravel for monitoring:

  1. Use a terminal and clone the following repository:

    https://github.com/unraveldata-org/unravel-terraform-scripts/tree/develop

    unravel-terraform-scripts.png

  2. Go to unravel-terraform-scripts/saas/bigquery folder.

  3. Duplicate the example input file named input.tfvars.example. Rename it to input.tfvars.

    cp input.tfvars.example input.tfvars

  4. Configure Terraform backend. Unravel recommends configuring a central storage for your Terraform state file.

    1. Copy the backend.tf.example file using this command:

      cp backend.tf.example backend.tf

    2. Update the Google Storage Path, to which the user who runs Terraform has access, in the backend.tf file.

  5. Using a VM editor, update the following in the input.tfvars file:

    bigquery-unravel-freemium-input_tfvars.png

    • Project IDs

      Enter the new project IDs that you had specified for exporting billing data in the monitoring_projects_list variable. You can specify up to 100 projects. In case you are upgrading from a Standard (Free) account, you can also specify the two project IDs that were set for the Standard (Free) account.

    • Administrator project ID

      Enter your administrator project ID in the admin_project_id variable. In case you are upgrading from the Standard (Free) account, you need not set the administrator project ID.

    • Service account

      Enter the service account that you received in the email after upgrading to Unravel Pro in the service_account variable.

  6. Navigate to the terraform directory and run the following commands: 

    cd bigquery-saas
terraform init
terraform plan --var-file=input.tfvars
terraform apply --var-file=input.tfvars

    The Terraform script generates a JSON file in the ./output folder. Keep this file handy when you integrate the BigQuery projects with Unravel.

In this section: