great_expectations.core

Package Contents

Classes

DataContextKey()

DataContextKey objects are used to uniquely identify resources used by the DataContext.

Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

DictDot()

RunIdentifier(run_name=None, run_time=None)

A RunIdentifier identifies a run (collection of validations) by run_name and run_time.

RunIdentifierSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

ExpectationConfiguration(expectation_type, kwargs, meta=None, success_on_last_run=None)

ExpectationConfiguration defines the parameters and name of a specific expectation.

ExpectationConfigurationSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

ExpectationSuite(expectation_suite_name, expectations=None, evaluation_parameters=None, data_asset_type=None, meta=None)

This ExpectationSuite object has create, read, update, and delete functionality for its expectations:

ExpectationSuiteSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

ExpectationValidationResult(success=None, expectation_config=None, result=None, meta=None, exception_info=None)

ExpectationValidationResultSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

ExpectationSuiteValidationResult(success=None, results=None, evaluation_parameters=None, statistics=None, meta=None)

ExpectationSuiteValidationResultSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

Functions

find_evaluation_parameter_dependencies(parameter_expression)

Parse a parameter expression to identify dependencies including GE URNs.

nested_update(d, u)

post_load(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after deserializing an object. The method

pre_dump(fn=None, pass_many=False)

Register a method to invoke before serializing an object. The method

in_jupyter_notebook()

get_metric_kwargs_id(metric_name, metric_kwargs)

convert_to_json_serializable(data)

Helper function to convert an object to one that is json serializable

ensure_json_serializable(data)

Helper function to convert an object to one that is json serializable

_deduplicate_evaluation_parameter_dependencies(dependencies)

great_expectations.core.ge_version
class great_expectations.core.DataContextKey

DataContextKey objects are used to uniquely identify resources used by the DataContext.

A DataContextKey is designed to support clear naming with multiple representations including a hashable version making it suitable for use as the key in a dictionary.

abstract to_tuple(self)
classmethod from_tuple(cls, tuple_)
abstract to_fixed_length_tuple(self)
abstract classmethod from_fixed_length_tuple(cls, tuple_)
__eq__(self, other)

Return self==value.

__ne__(self, other)

Return self!=value.

__hash__(self)

Return hash(self).

__repr__(self)

Return repr(self).

great_expectations.core.find_evaluation_parameter_dependencies(parameter_expression)

Parse a parameter expression to identify dependencies including GE URNs.

Parameters

parameter_expression – the parameter to parse

Returns

  • “urns”: set of strings that are valid GE URN objects

  • ”other”: set of non-GE URN strings that are required to evaluate the parameter expression

Return type

a dictionary including

great_expectations.core.ge_urn
great_expectations.core.nested_update(d, u)
exception great_expectations.core.DataContextError(message)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

exception great_expectations.core.InvalidCacheValueError(result_dict)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

exception great_expectations.core.InvalidExpectationConfigurationError(message)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

exception great_expectations.core.InvalidExpectationKwargsError(message)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

exception great_expectations.core.ParserError(message)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

exception great_expectations.core.UnavailableMetricError(message)

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

class great_expectations.core.Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.base.SchemaABC

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

class Meta

Options object for a Schema.

Example usage:

class Meta:
    fields = ("id", "email", "date_created")
    exclude = ("password", "secret_attribute")

Available options:

  • fields: Tuple or list of fields to include in the serialized result.

  • additional: Tuple or list of fields to include in addition to the

    explicitly declared fields. additional and fields are mutually-exclusive options.

  • include: Dictionary of additional fields to include in the schema. It is

    usually better to define fields as class variables, but you may need to use this option, e.g., if your fields are Python keywords. May be an OrderedDict.

  • exclude: Tuple or list of fields to exclude in the serialized result.

    Nested fields can be represented with dot delimiters.

  • dateformat: Default format for Date <fields.Date> fields.

  • datetimeformat: Default format for DateTime <fields.DateTime> fields.

  • render_module: Module to use for loads <Schema.loads> and dumps <Schema.dumps>.

    Defaults to json from the standard library.

  • ordered: If True, order serialization output according to the

    order in which fields were declared. Output of Schema.dump will be a collections.OrderedDict.

  • index_errors: If True, errors dictionaries will include the index

    of invalid items in a collection.

  • load_only: Tuple or list of fields to exclude from serialized results.

  • dump_only: Tuple or list of fields to exclude from deserialization

  • unknown: Whether to exclude, include, or raise an error for unknown

    fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • register: Whether to register the Schema with marshmallow’s internal

    class registry. Must be True if you intend to refer to this Schema by class name in Nested fields. Only set this to False when memory usage is critical. Defaults to True.

TYPE_MAPPING :typing.Dict[type, typing.Type[ma_fields.Field]]
error_messages :typing.Dict[str, str]
_default_error_messages :typing.Dict[str, str]
OPTIONS_CLASS :type
opts :SchemaOpts
_declared_fields :typing.Dict[str, ma_fields.Field]
_hooks :typing.Dict[types.Tag, typing.List[str]]
__repr__(self)

Return repr(self).

property dict_class(self)
property set_class(self)
classmethod from_dict(cls, fields: typing.Dict[str, typing.Union[ma_fields.Field, type]], *, name: str = 'GeneratedSchema')

Generate a Schema class given a dictionary of fields.

from great_expectations.marshmallow__shade import Schema, fields

PersonSchema = Schema.from_dict({"name": fields.Str()})
print(PersonSchema().load({"name": "David"}))  # => {'name': 'David'}

Generated schemas are not added to the class registry and therefore cannot be referred to by name in Nested fields.

Parameters
  • fields (dict) – Dictionary mapping field names to field instances.

  • name (str) – Optional name for the class, which will appear in the repr for the class.

New in version 3.0.0.

handle_error(self, error: ValidationError, data: typing.Any, *, many: bool, **kwargs)

Custom error handler function for the schema.

Parameters
  • error – The ValidationError raised during (de)serialization.

  • data – The original input data.

  • many – Value of many on dump or load.

  • partial – Value of partial on load.

New in version 2.0.0.

Changed in version 3.0.0rc9: Receives many and partial (on deserialization) as keyword arguments.

get_attribute(self, obj: typing.Any, attr: str, default: typing.Any)

Defines how to pull values from an object to serialize.

New in version 2.0.0.

Changed in version 3.0.0a1: Changed position of obj and attr.

static _call_and_store(getter_func, data, *, field_name, error_store, index=None)

Call getter_func with data as its argument, and store any ValidationErrors.

Parameters
  • getter_func (callable) – Function for getting the serialized/deserialized value from data.

  • data – The data passed to getter_func.

  • field_name (str) – Field name.

  • index (int) – Index of the item being validated, if validating a collection, otherwise None.

_serialize(self, obj: typing.Union[_T, typing.Iterable[_T]], *, many: bool = False)

Serialize obj.

Parameters
  • obj – The object(s) to serialize.

  • many (bool) – True if data should be serialized as a collection.

Returns

A dictionary of the serialized data

Changed in version 1.0.0: Renamed from marshal.

dump(self, obj: typing.Any, *, many: bool = None)

Serialize an object to native Python data types according to this Schema’s fields.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A dict of serialized data

Return type

dict

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

Changed in version 3.0.0rc9: Validation no longer occurs upon serialization.

dumps(self, obj: typing.Any, *args, many: bool = None, **kwargs)

Same as dump(), except return a JSON-encoded string.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A json string

Return type

str

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

_deserialize(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, error_store: ErrorStore, many: bool = False, partial=False, unknown=RAISE, index=None)

Deserialize data.

Parameters
  • data (dict) – The data to deserialize.

  • error_store (ErrorStore) – Structure to store errors.

  • many (bool) – True if data should be deserialized as a collection.

  • partial (bool|tuple) – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • index (int) – Index of the item being serialized (for storing errors) if serializing a collection, otherwise None.

Returns

A dictionary of the deserialized data.

load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None)

Deserialize a data structure to an object defined by this Schema’s fields.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

loads(self, json_data: str, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, **kwargs)

Same as load(), except it takes a JSON string as input.

Parameters
  • json_data – A JSON string of the data to deserialize.

  • many – Whether to deserialize obj as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

_run_validator(self, validator_func, output, *, original_data, error_store, many, partial, pass_original, index=None)
validate(self, data: typing.Mapping, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None)

Validate data against the schema, returning a dictionary of validation errors.

Parameters
  • data – The data to validate.

  • many – Whether to validate data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

Returns

A dictionary of validation errors.

New in version 1.1.0.

_do_load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, postprocess: bool = True)

Deserialize data, returning the deserialized result. This method is private API.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to validate required fields. If its value is an iterable, only fields listed in that iterable will be ignored will be allowed missing. If True, all fields will be allowed missing. If None, the value for self.partial is used.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

  • postprocess – Whether to run post_load methods..

Returns

Deserialized data

_normalize_nested_options(self)

Apply then flatten nested schema options. This method is private API.

__apply_nested_option(self, option_name, field_names, set_operation)

Apply nested options to nested fields

_init_fields(self)

Update self.fields, self.load_fields, and self.dump_fields based on schema options. This method is private API.

on_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Hook to modify a field when it is bound to the Schema.

No-op by default.

_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Bind field to the schema, setting any necessary attributes on the field (e.g. parent and name).

Also set field load_only and dump_only values if field_name was specified in class Meta.

_has_processors(self, tag)
_invoke_dump_processors(self, tag: str, data, *, many: bool, original_data=None)
_invoke_load_processors(self, tag: str, data, *, many: bool, original_data, partial: typing.Union[bool, types.StrSequenceOrSet])
_invoke_field_validators(self, *, error_store: ErrorStore, data, many: bool)
_invoke_schema_validators(self, *, error_store: ErrorStore, pass_many: bool, data, original_data, many: bool, partial: typing.Union[bool, types.StrSequenceOrSet], field_errors: bool = False)
_invoke_processors(self, tag: str, *, pass_many: bool, data, many: bool, original_data=None, **kwargs)
exception great_expectations.core.ValidationError(message: typing.Union[str, typing.List, typing.Dict], field_name: str = SCHEMA, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]] = None, valid_data: typing.Union[typing.List[typing.Dict[str, typing.Any]], typing.Dict[str, typing.Any]] = None, **kwargs)

Bases: great_expectations.marshmallow__shade.exceptions.MarshmallowError

Raised when validation fails on a field or schema.

Validators and custom fields should raise this exception.

Parameters
  • message – An error message, list of error messages, or dict of error messages. If a dict, the keys are subitems and the values are error messages.

  • field_name – Field name to store the error on. If None, the error is stored as schema-level error.

  • data – Raw input data.

  • valid_data – Valid (de)serialized data.

normalized_messages(self)
great_expectations.core.post_load(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after deserializing an object. The method receives the deserialized data and returns the processed data.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s load() call. If pass_many=True, the raw data (which may be a collection) is passed.

If pass_original=True, the original data (before deserializing) will be passed as an additional argument to the method.

Changed in version 3.0.0: partial and many are always passed as keyword arguments to the decorated method.

great_expectations.core.pre_dump(fn=None, pass_many=False)

Register a method to invoke before serializing an object. The method receives the object to be serialized and returns the processed object.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s dump() call. If pass_many=True, the raw data (which may be a collection) is passed.

Changed in version 3.0.0: many is always passed as a keyword arguments to the decorated method.

class great_expectations.core.DictDot
__getitem__(self, item)
__setitem__(self, key, value)
__delitem__(self, key)
great_expectations.core.logger
great_expectations.core.RESULT_FORMATS = ['BOOLEAN_ONLY', 'BASIC', 'COMPLETE', 'SUMMARY']
great_expectations.core.EvaluationParameterIdentifier
great_expectations.core.in_jupyter_notebook()
great_expectations.core.get_metric_kwargs_id(metric_name, metric_kwargs)
great_expectations.core.convert_to_json_serializable(data)

Helper function to convert an object to one that is json serializable

Parameters

data – an object to attempt to convert a corresponding json-serializable object

Returns

(dict) A converted test_object

Warning

test_obj may also be converted in place.

great_expectations.core.ensure_json_serializable(data)

Helper function to convert an object to one that is json serializable

Parameters

data – an object to attempt to convert a corresponding json-serializable object

Returns

(dict) A converted test_object

Warning

test_obj may also be converted in place.

class great_expectations.core.RunIdentifier(run_name=None, run_time=None)

Bases: great_expectations.core.data_context_key.DataContextKey

A RunIdentifier identifies a run (collection of validations) by run_name and run_time.

property run_name(self)
property run_time(self)
to_tuple(self)
to_fixed_length_tuple(self)
__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

to_json_dict(self)
classmethod from_tuple(cls, tuple_)
classmethod from_fixed_length_tuple(cls, tuple_)
class great_expectations.core.RunIdentifierSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.Schema

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

run_name
run_time
make_run_identifier(self, data, **kwargs)
great_expectations.core._deduplicate_evaluation_parameter_dependencies(dependencies)
class great_expectations.core.ExpectationConfiguration(expectation_type, kwargs, meta=None, success_on_last_run=None)

Bases: great_expectations.types.DictDot

ExpectationConfiguration defines the parameters and name of a specific expectation.

kwarg_lookup_dict
runtime_kwargs = ['result_format', 'include_config', 'catch_exceptions']
patch(self, op: str, path: str, value: Any)
Parameters
  • op – A jsonpatch operation. One of ‘add’, ‘replace’, or ‘remove’

  • path – A jsonpatch path for the patch operation

  • value – The value to patch

Returns

The patched ExpectationConfiguration object

property expectation_type(self)
property kwargs(self)
_get_default_custom_kwargs(self)
get_domain_kwargs(self)
get_success_kwargs(self)
get_runtime_kwargs(self)
applies_to_same_domain(self, other_expectation_configuration)
isEquivalentTo(self, other, match_type='success')

ExpectationConfiguration equivalence does not include meta, and relies on equivalence of kwargs.

__eq__(self, other)

ExpectationConfiguration equality does include meta, but ignores instance identity.

__ne__(self, other)

Return self!=value.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

to_json_dict(self)
get_evaluation_parameter_dependencies(self)
class great_expectations.core.ExpectationConfigurationSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.Schema

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

expectation_type
kwargs
meta
make_expectation_configuration(self, data, **kwargs)
class great_expectations.core.ExpectationSuite(expectation_suite_name, expectations=None, evaluation_parameters=None, data_asset_type=None, meta=None)

This ExpectationSuite object has create, read, update, and delete functionality for its expectations: -create: self.add_expectation() -read: self.find_expectation_indexes() -update: self.add_expectation() or self.patch_expectation() -delete: self.remove_expectation()

add_citation(self, comment, batch_kwargs=None, batch_markers=None, batch_parameters=None, citation_date=None)
isEquivalentTo(self, other)

ExpectationSuite equivalence relies only on expectations and evaluation parameters. It does not include: - data_asset_name - expectation_suite_name - meta - data_asset_type

__eq__(self, other)

ExpectationSuite equality ignores instance identity, relying only on properties.

__ne__(self, other)

Return self!=value.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

to_json_dict(self)
get_evaluation_parameter_dependencies(self)
get_citations(self, sort=True, require_batch_kwargs=False)
get_table_expectations(self)

Return a list of table expectations.

get_column_expectations(self)

Return a list of column map expectations.

static _filter_citations(citations, filter_key)
static _sort_citations(citations)
append_expectation(self, expectation_config)

Appends an expectation.

Parameters

expectation_config (ExpectationConfiguration) – The expectation to be added to the list.

Notes

May want to add type-checking in the future.

remove_expectation(self, expectation_configuration: ExpectationConfiguration, match_type: str = 'domain', remove_multiple_matches: bool = False)
Parameters
  • expectation_configuration – A potentially incomplete (partial) Expectation Configuration to match against for for the removal of expectations.

  • match_type

    This determines what kwargs to use when matching. Options are ‘domain’ to match based on the data evaluated by that expectation, ‘success’ to match based on all configuration parameters

    that influence whether an expectation succeeds based on a given batch of data, and ‘runtime’ to match based on all configuration parameters

  • remove_multiple_matches – If True, will remove multiple matching expectations. If False, will raise a ValueError.

Returns: The list of deleted ExpectationConfigurations

Raises
  • No match

  • More than 1 match, if remove_multiple_matches = False

remove_all_expectations_of_type(self, expectation_types: Union[List[str], str])
find_expectation_indexes(self, expectation_configuration: ExpectationConfiguration, match_type: str = 'domain')
Parameters
  • expectation_configuration – A potentially incomplete (partial) Expectation Configuration to match against to find the index of any matching Expectation Configurations on the suite.

  • match_type

    This determines what kwargs to use when matching. Options are ‘domain’ to match based on the data evaluated by that expectation, ‘success’ to match based on all configuration parameters

    that influence whether an expectation succeeds based on a given batch of data, and ‘runtime’ to match based on all configuration parameters

Returns: A list of indexes of matching ExpectationConfiguration

Raises

InvalidExpectationConfigurationError

find_expectations(self, expectation_configuration: ExpectationConfiguration, match_type: str = 'domain')
patch_expectation(self, expectation_configuration: ExpectationConfiguration, op: str, path: str, value: Any, match_type: str)
Parameters
  • expectation_configuration – A potentially incomplete (partial) Expectation Configuration to match against to find the expectation to patch.

  • op – A jsonpatch operation (one of ‘add’,’update’, or ‘remove’) (see http://jsonpatch.com/)

  • path – A jsonpatch path for the patch operation (see http://jsonpatch.com/)

  • value – The value to patch (see http://jsonpatch.com/)

  • match_type – The match type to use for find_expectation_index()

Returns: The patched ExpectationConfiguration

Raises
  • No match

  • More than 1 match

add_expectation(self, expectation_configuration: ExpectationConfiguration, match_type: str = 'domain', overwrite_existing: bool = True)
Parameters
  • expectation_configuration – The ExpectationConfiguration to add or update

  • match_type – The criteria used to determine whether the Suite already has an ExpectationConfiguration and so whether we should add or replace.

  • overwrite_existing – If the expectation already exists, this will overwrite if True and raise an error if False.

Returns

The ExpectationConfiguration to add or replace.

Raises
  • More than one match

  • One match if overwrite_existing = False

class great_expectations.core.ExpectationSuiteSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.Schema

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

expectation_suite_name
expectations
evaluation_parameters
data_asset_type
meta
clean_empty(self, data)
prepare_dump(self, data, **kwargs)
make_expectation_suite(self, data, **kwargs)
class great_expectations.core.ExpectationValidationResult(success=None, expectation_config=None, result=None, meta=None, exception_info=None)
__eq__(self, other)

ExpectationValidationResult equality ignores instance identity, relying only on properties.

__ne__(self, other)

Return self!=value.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

validate_result_dict(self, result)
to_json_dict(self)
get_metric(self, metric_name, **kwargs)
class great_expectations.core.ExpectationValidationResultSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.Schema

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

success
expectation_config
result
meta
exception_info
convert_result_to_serializable(self, data, **kwargs)
make_expectation_validation_result(self, data, **kwargs)
class great_expectations.core.ExpectationSuiteValidationResult(success=None, results=None, evaluation_parameters=None, statistics=None, meta=None)

Bases: great_expectations.types.DictDot

__eq__(self, other)

ExpectationSuiteValidationResult equality ignores instance identity, relying only on properties.

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

to_json_dict(self)
get_metric(self, metric_name, **kwargs)
class great_expectations.core.ExpectationSuiteValidationResultSchema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.Schema

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

success
results
evaluation_parameters
statistics
meta
prepare_dump(self, data, **kwargs)
make_expectation_suite_validation_result(self, data, **kwargs)
great_expectations.core.expectationConfigurationSchema
great_expectations.core.expectationSuiteSchema
great_expectations.core.expectationValidationResultSchema
great_expectations.core.expectationSuiteValidationResultSchema
great_expectations.core.runIdentifierSchema