Validation Operators Module

ValidationOperator

class great_expectations.validation_operators.ValidationOperator

Bases: object

The base class of all validation operators.

It defines the signature of the public run method - this is the only contract re operators’ API. Everything else is up to the implementors of validation operator classes that will be the descendants of this base class.

run(assets_to_validate, run_id)

ActionListValidationOperator

class great_expectations.validation_operators.ActionListValidationOperator(data_context, action_list)

Bases: great_expectations.validation_operators.validation_operators.ValidationOperator

ActionListValidationOperator is a validation operator that validates each batch in the list that is passed to its run method and then invokes a list of configured actions on every validation result.

A user can configure the list of actions to invoke.

Each action in the list must be an instance of NamespacedValidationAction class (or its descendants).

Below is an example of this operator’s configuration:

action_list_operator:
    class_name: ActionListValidationOperator
    action_list:
      - name: store_validation_result
        action:
          class_name: StoreAction
          target_store_name: validations_store
      - name: store_evaluation_params
        action:
          class_name: ExtractAndStoreEvaluationParamsAction
          target_store_name: evaluation_parameter_store
      - name: send_slack_notification_on_validation_result
        action:
          class_name: SlackNotificationAction
          # put the actual webhook URL in the uncommitted/config_variables.yml file
          slack_webhook: ${validation_notification_slack_webhook}
         notify_on: all # possible values: "all", "failure", "success"
          renderer:
            module_name: great_expectations.render.renderer.slack_renderer
            class_name: SlackRenderer
run(assets_to_validate, run_id)

WarningAndFailureExpectationSuitesValidationOperator

class great_expectations.validation_operators.WarningAndFailureExpectationSuitesValidationOperator(data_context, action_list, expectation_suite_name_prefix='', expectation_suite_name_suffixes=['failure', 'warning'], stop_on_first_error=False, slack_webhook=None, notify_on='all')

Bases: great_expectations.validation_operators.validation_operators.ActionListValidationOperator

WarningAndFailureExpectationSuitesValidationOperator is a validation operator that accepts a list batches of data assets (or the information necessary to fetch these batches). The operator retrieves 2 expectation suites for each data asset/batch - one containing the critical expectations (“failure”) and the other containing non-critical expectations (“warning”). By default, the operator assumes that the first is called “failure” and the second is called “warning”, but “expectation_suite_name_prefix” attribute can be specified in the operator’s configuration to make sure it searched for “{expectation_suite_name_prefix}failure” and {expectation_suite_name_prefix}warning” expectation suites for each data asset.

The operator validates each batch against its “failure” and “warning” expectation suites and invokes a list of actions on every validation result.

The list of these actions is specified in the operator’s configuration

Each action in the list must be an instance of NamespacedValidationAction class (or its descendants).

The operator sends a Slack notification (if “slack_webhook” is present in its config). The “notify_on” config property controls whether the notification should be sent only in the case of failure (“failure”), only in the case of success (“success”), or always (“all”).

Below is an example of this operator’s configuration:

run_warning_and_failure_expectation_suites:
    class_name: WarningAndFailureExpectationSuitesValidationOperator
    # put the actual webhook URL in the uncommitted/config_variables.yml file
    slack_webhook: ${validation_notification_slack_webhook}
    action_list:
      - name: store_validation_result
        action:
          class_name: StoreAction
          target_store_name: validations_store
      - name: store_evaluation_params
        action:
          class_name: ExtractAndStoreEvaluationParamsAction
          target_store_name: evaluation_parameter_store

The operator returns an object that looks like the example below.

The value of “success” is True if no critical expectation suites (“failure”) failed to validate (non-critial (“warning”) expectation suites are allowed to fail without affecting the success status of the run:

{
    "data_asset_identifiers": [list, of, data, asset, identifiers],
    "success": True/False,
    "failure": {
        "expectation_suite_identifier": {
            "validation_result": validation_result,
            "action_results": {
                "action name": "action result object"
            }
        }
    },
    "warning": {
        "expectation_suite_identifier": {
            "validation_result": validation_result,
            "action_results": {
                "action name": "action result object"
            }
        }
    }
}
run(assets_to_validate, run_id)