Configuring Bigquery Data Page
You can configure the Data page for BigQuery when you install and set up Unravel for BigQuery monitoring.
Create a role for an Unravel project with the required permission for Unravel monitoring. This role is assigned to the Unravel service account that is used to authenticate Unravel applications.
On the GCP Console, go to the Roles page.
Using the drop-down list at the top of the page, select the Unravel project in which you want to create a role.
Click Create Role and enter a Name, Title, and Description for the role. Preferably keep the role name as unravel for easy identification. The description is optional, which can be maintained as Unravel monitoring.
Select Role launch stage as General Availability.
Caution
The role name cannot be changed after the role is created.
Click Add Permissions.
In the Add Permissions dialog box, filter and select the following permissions for the role, and then click Add.
Note
These are permissions are only for fetching data from BigQuery to Unravel Data page.
Permission
Description
bigquery.datasets.get
bigquery.routines.get
bigquery.routines.list
bigquery.tables.getData
bigquery.tables.list
Permissions to get the tables and partitions metadata. These permissions are required for the Data page.
bigquery.jobs.create
Permissions to execute queries on BigQuery to fetch the metadata about the tables and partitions. These permissions are required for the Data page.
Click Create. The role is created.
Refer to you Install and set up Unravel for BigQuery monitoring to complete the setup process for BigQuery, if not done already.
Queries executed by Unravel to fetch table and partitions details
Unravel executes the following queries to fetch the table and partition details. These queries are charged based on the pricing model of BigQuery. Bigquery charges for a minimum of 10 MB of data processed for the following queries. However, if the data processing is more than 10 MB it will be charged accordingly. More information about the pricing can be found here.
Query to fetch table information
SELECT t.dataset_id, t.table_id, t.creation_time, t.last_modified_time, t.row_count,t.size_bytes, t.type,i.table_type,i.ddl FROM `<project_id>.<dataset_id>.__TABLES__` AS t JOIN `<project_id>.<dataset_id>.INFORMATION_SCHEMA.TABLES` AS i ON t.table_id = i.table_name and t.dataset_id = i.table_schema;
Query to fetch partition information
SELECT table_catalog,table_schema,table_name,partition_id, total_rows,total_logical_bytes, total_billable_bytes, last_modified_time, storage_tier FROM `<project_id>.<dataset_id>.INFORMATION_SCHEMA.PARTITIONS`;