Skip to main content
Version: 0.18.9

Add Validation data or Expectation Suites to a Checkpoint

Add validation data or Expectation SuitesA collection of verifiable assertions about data. to an existing CheckpointThe primary means for validating data in a production deployment of Great Expectations. to aggregate individual validations across Expectation Suites or Data SourcesProvides a standard API for accessing and interacting with data from a wide variety of source systems. into a single Checkpoint. You can also use this process to ValidateThe act of applying an Expectation Suite to a Batch. multiple source files before and after their ingestion into your data lake.

Prerequisites

Add an Expectation Suite to the Checkpoint

To add a second Expectation Suite (in the following example, users.error) to your Checkpoint configuration, you update the Validations in your Checkpoint. The configuration appears similar to this example:

Python
validations = [
{
"batch_request": {
"datasource_name": "my_datasource",
"data_asset_name": "users",
},
"expectation_suite_name": "users.warning",
},
{
"batch_request": {
"datasource_name": "my_datasource",
"data_asset_name": "users.error",
},
"expectation_suite_name": "users",
},
]

checkpoint = context.add_or_update_checkpoint(
name="my_test_checkpoint",
validations=validations,
)

Add Validation data to the Checkpoint

In the previous example, you added a Validation and it was paired with the same Batch as the original Expectation Suite. However, you can also specify different Batch RequestsProvided to a Data Source in order to create a Batch. when you add an Expectation Suite. Adding multiple Validations with different Expectation Suites and specific ActionsA Python class with a run method that takes a Validation Result and does something with it is shown in the following example:

Python
validations = [
{
"batch_request": {
"datasource_name": "my_datasource",
"data_asset_name": "users",
},
"expectation_suite_name": "users.warning",
},
{
"batch_request": {
"datasource_name": "my_datasource",
"data_asset_name": "users",
},
"expectation_suite_name": "users.error",
},
{
"batch_request": {
"datasource_name": "my_datasource",
"data_asset_name": "users",
},
"expectation_suite_name": "users.delivery",
"action_list": [
{
"name": "quarantine_failed_data",
"action": {
"class_name": "CreateQuarantineData",
},
},
{
"name": "advance_passed_data",
"action": {
"class_name": "CreateQuarantineData",
},
},
],
},
]

checkpoint = context.add_or_update_checkpoint(
name="my_test_checkpoint", validations=validations
)

In this Checkpoint configuration, the Expectation Suite users.warning runs against the batch_request and the results are processed by the Actions specified in the action_list. Similarly, the Expectation Suite users.error runs against the batch_request and the results processed by the actions specified in the action_list. In addition, the Expectation Suite users.delivery runs against the batch_request and the results are processed by the actions in the Validation action_list and its action_list.

For additional Checkpoint configuration information, see Manage Checkpoints.