Skip to main content

Home

BigQuery GE integration

Prerequisites
Integrating Great Expectations (BigQuery)

Run the following steps to integrate GE for BigQuery Data page.

  1. Setup Python Environment.

    1. Download the .whl file for the Unravel contrib package.

    2. Open a terminal window and navigate to the directory where the .whl file is located.

    3. Run the following command to install the Unravel contrib package:

      pip install <file_name.whl>

  2. Configure Unravelaction.

    The unravelaction has the capability to extract events out of the expectation failures generated by the GE test suite.

    1. In the Python file where the checkpoint is defined, import unravelaction:

      import unravelaction

    2. Configure unravelaction to the action_list configuration in the GE checkpoint YAML configuration. 

      action_list:
  - name: UnravelAction
    action:
      class_name: UnravelAction
      module_name: unravelaction
      lr_url: "{lr_url}"
      lr_version: "v2"
      index: "events_bq_t1-"

      The configurable parameters are as follows:

      name

      Action name for storing validation results.

      class_name

      Set it to UnravelAction.

      module_name

      Set it to unravelaction.

      lr_url

      URL to the log receiver.

      lr_version

      The version of the log receiver.

      index

      The Index name where the GE event will be stored.

  3. Run the Great Expectations checkpoint.

    To validate the data and send the expectation failures to unravel, you need to run the checkpoint. You can check the great expectations guide on different ways to run a checkpoint.

In this section: