Skip to main content
Version: 0.18.21

Edit an existing Expectation Suite

Use the information provided here to learn how to edit an Expectation Suite. Editing Expectations does not edit or alter the Batch data.

All the code used in the examples is available in GitHub at this location: how_to_edit_an_expectation_suite.py.

Prerequisites

    • A working installation of Great Expectations
    • A Filesystem Data Context for your Expectations
    • A Data Source from which to request a Batch of data for introspection
    • An Expectation Suite

    Import the Great Expectations module and instantiate a Data Context

    Run the following code to create a new Data Context with the get_context() method:

    Python
    import great_expectations as gx
    import great_expectations.expectations as gxe

    context = gx.get_context()

    Create a Validator from Data

    Run the following code to connect to .csv data stored in the great_expectations GitHub repository:

    Python
    validator = context.sources.pandas_default.read_csv(
    "https://raw.githubusercontent.com/great-expectations/gx_tutorials/main/data/yellow_tripdata_sample_2019-01.csv"
    )

    Retrieve an existing Expectation Suite

    Run the following code to retrieve an Expectation Suite:

    my suite = context.get_expectation_suite("expectation_suite_name")

    Replace expectation_suite_name with the name of your Expectation Suite.

    View the Expectations in the Expectation Suite

    Run the following code to print the Suite to console or in a Python interpreter the show_expectations_by_expectation_type() method:

    Python
    my_suite.show_expectations_by_expectation_type()

    The output appears similar to the following example:

        [ { 'expect_column_values_to_be_between': { 'auto': True,
    'column': 'passenger_count',
    'domain': 'column',
    'max_value': 6,
    'min_value': 1,
    'mostly': 1.0,
    'strict_max': False,
    'strict_min': False}},
    { 'expect_column_values_to_not_be_null': { 'column': 'pickup_datetime',
    'domain': 'column'}}]

    Instantiate ExpectationConfiguration

    From the Expectation Suite, you can create an ExpectationConfiguration object using the output from `show_expectations_by_expectation_type(). The following is the example output from the first Expectation in the Expectation Suite.

    It runs the expect_column_values_to_be_between Expectation on the passenger_count column and expects the min and max values to be 1 and 6 respectively.

    Python
    {
    "expect_column_values_to_be_between": {
    "column": "passenger_count",
    "max_value": 6,
    "min_value": 1,
    "mostly": 1.0,
    "strict_max": False,
    "strict_min": False,
    }
    }

    The following is the same configuration with an ExpectationConfiguration object:

    Python
    from great_expectations.expectations.expectation_configuration import (
    ExpectationConfiguration,
    )
    Python
    config = ExpectationConfiguration(
    expectation_type="expect_column_values_to_be_between",
    kwargs={
    "column": "passenger_count",
    "max_value": 6,
    "min_value": 1,
    "mostly": 1.0,
    "strict_max": False,
    "strict_min": False,
    },
    )

    Update the Configuration and Expectation Suite

    In the following example, the max_value of the Expectation is adjusted from 4 to 6 with a new ExpectationConfiguration:

    Python
    updated_config = ExpectationConfiguration(
    expectation_type="expect_column_values_to_be_between",
    kwargs={
    "column": "passenger_count",
    "min_value": 1,
    "max_value": 4,
    #'max_value': 6,
    "mostly": 1.0,
    "strict_max": False,
    "strict_min": False,
    },
    )

    To update the Expectation Suite you use the add_expectation() function. For example:

    Python
    my_suite.add_expectation_configuration(updated_config)

    The add_expectation() function performs an 'upsert' into the ExpectationSuite and updates the existing Expectation, or adds a new one if it doesn't.

    To check that the Expectation Suite has been updated, you can run the show_expectations_by_expectation_type() function again, or run find_expectation() and then confirm that the expected Expectation exists in the suite. For example:

    Python
    config_to_search = ExpectationConfiguration(
    expectation_type="expect_column_values_to_be_between",
    kwargs={"column": "passenger_count"},
    )
    found_expectation = my_suite.find_expectations(config_to_search, match_type="domain")

    # This assertion will succeed because the ExpectationConfiguration has been updated.
    assert len(found_expectation) == 1

    You'll need to perform the search with a new ExpectationConfiguration, but you don't need to include all the kwarg values.

    Remove the ExpectationConfiguration (Optional)

    To remove an ExpectationConfiguration, you can use the remove_configuration() function. Similar to find_expectation(), you call the remove_configuration() function with ExpectationConfiguration. For example:

    Python
    config_to_remove = ExpectationConfiguration(
    expectation_type="expect_column_values_to_be_between",
    kwargs={"column": "passenger_count"},
    )
    my_suite.remove_expectation(
    config_to_remove, match_type="domain", remove_multiple_matches=False
    )

    found_expectation = my_suite.find_expectations(config_to_remove, match_type="domain")

    # This assertion will fail because the ExpectationConfiguration has been removed.
    assert found_expectation != [updated_config]
    my_suite.show_expectations_by_expectation_type()

    The output of show_expectations_by_expectation_type() should appear similar to this example:

    [ 
    { 'expect_column_values_to_not_be_null': { 'column': 'pickup_datetime',
    'domain': 'column'}}]

    Save Expectation Suite changes

    After editing an Expectation Suite, you can use the save_suite() function to save it to your Data Context. For example:

    Python
    context.save_expectation_suite(my_suite)

    To make sure your Expectation Suite changes are reflected in the Validator, use context.get_validator() to overwrite the validator, or create a new one from the updated Data Context.