How to add validations, data, or suites to a Checkpoint

This guide will help you add validations, data or suites to an existing Checkpoint. This is useful if you want to aggregate individual validations (across suites or datasources) into a single Checkpoint.

Prerequisites: This how-to guide assumes you have already:

Steps

1. First, open your existing Checkpoint in a text editor. It will look similar to this:

validation_operator_name: action_list_operator
batches:
  - batch_kwargs:
      path: /home/me/my_project/source_files/npi.csv
      datasource: files_datasource
      reader_method: read_csv
    expectation_suite_names:
      - npi.warning
  1. To add a second suite (in this example we add npi.critical) to your Checkpoint modify the file to look like this:

validation_operator_name: action_list_operator
batches:
  - batch_kwargs:
      path: /home/me/my_project/source_files/npi.csv
      datasource: files_datasource
      reader_method: read_csv
    expectation_suite_names:
      - npi.warning
      - npi.critical
  1. To add a second validation of a batch of data (in this case a table named npi from a datasource named data_lake) to your Checkpoint modify the file to look like this:

validation_operator_name: action_list_operator
batches:
  - batch_kwargs:
      path: /home/me/my_project/source_files/npi.csv
      datasource: files_datasource
      reader_method: read_csv
    expectation_suite_names:
      - npi.warning
      - another_suite
  - batch_kwargs:
      table: npi
      datasource: data_lake
    expectation_suite_names:
      - npi.warning

Additional notes

Tip

This is a good way to aggregate validations in a complex pipeline. You could use this feature to validate multiple source files before and after their ingestion into your data lake.

If it would be useful to you, please comment with a +1 and feel free to add any suggestions or questions below.

If you want to be a real hero, we’d welcome a pull request. Please see the Contributing tutorial and How to write a how to guide to get started.