great_expectations.data_context.store

Package Contents

Classes

DatabaseStoreBackend(credentials, table_name, key_columns, fixed_length_key=True)

A store backend acts as a key-value store that can accept tuples as keys, to abstract away

ExpectationsStore(store_backend=None, runtime_environment=None)

A store is responsible for reading and writing Great Expectations objects

HtmlSiteStore(store_backend=None, runtime_environment=None)

EvaluationParameterStore(store_backend=None)

A MetricStore stores ValidationMetric information to be used between runs.

MetricStore(store_backend=None)

A MetricStore stores ValidationMetric information to be used between runs.

Store(store_backend=None, runtime_environment=None)

A store is responsible for reading and writing Great Expectations objects

InMemoryStoreBackend(runtime_environment=None, fixed_length_key=False)

Uses an in-memory dictionary as a store backend.

StoreBackend(fixed_length_key=False)

A store backend acts as a key-value store that can accept tuples as keys, to abstract away

TupleFilesystemStoreBackend(base_directory, filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=True, root_directory=None, fixed_length_key=False)

Uses a local filepath as a store.

TupleGCSStoreBackend(bucket, project, prefix=’‘, filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=False, fixed_length_key=False, public_urls=True)

Uses a GCS bucket as a store.

TupleS3StoreBackend(bucket, prefix=’‘, filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=False, fixed_length_key=False)

Uses an S3 bucket as a store.

TupleStoreBackend(filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=True, fixed_length_key=False)

If filepath_template is provided, the key to this StoreBackend abstract class must be a tuple with

ValidationsStore(store_backend=None, runtime_environment=None)

A store is responsible for reading and writing Great Expectations objects

class great_expectations.data_context.store.DatabaseStoreBackend(credentials, table_name, key_columns, fixed_length_key=True)

Bases: great_expectations.data_context.store.store_backend.StoreBackend

A store backend acts as a key-value store that can accept tuples as keys, to abstract away reading and writing to a persistence layer.

In general a StoreBackend implementation must provide implementations of:
  • _get

  • _set

  • list_keys

  • _has_key

_get(self, key)
_set(self, key, value, **kwargs)
abstract _move(self)
_has_key(self, key)
list_keys(self, prefix=())
remove_key(self, key)
class great_expectations.data_context.store.ExpectationsStore(store_backend=None, runtime_environment=None)

Bases: great_expectations.data_context.store.store.Store

A store is responsible for reading and writing Great Expectations objects to appropriate backends. It provides a generic API that the DataContext can use independently of any particular ORM and backend.

An implementation of a store will generally need to define the following:
  • serialize

  • deserialize

  • _key_class (class of expected key type)

All keys must have a to_tuple() method.

_key_class
remove_key(self, key)
serialize(self, key, value)
deserialize(self, key, value)
class great_expectations.data_context.store.HtmlSiteStore(store_backend=None, runtime_environment=None)

Bases: object

_key_class
get(self, key)
set(self, key, serialized_value)
get_url_for_resource(self, resource_identifier=None, only_if_exists=True)

Return the URL of the HTML document that renders a resource (e.g., an expectation suite or a validation result).

Parameters

resource_identifier – ExpectationSuiteIdentifier, ValidationResultIdentifier or any other type’s identifier. The argument is optional - when not supplied, the method returns the URL of the index page.

Returns

URL (string)

_validate_key(self, key)
list_keys(self)
write_index_page(self, page)

This third param_store has a special method, which uses a zero-length tuple as a key.

clean_site(self)
copy_static_assets(self, static_assets_source_dir=None)

Copies static assets, using a special “static_assets” backend store that accepts variable-length tuples as keys, with no filepath_template.

class great_expectations.data_context.store.EvaluationParameterStore(store_backend=None)

Bases: great_expectations.data_context.store.metric_store.MetricStore

A MetricStore stores ValidationMetric information to be used between runs.

get_bind_params(self, run_id)
class great_expectations.data_context.store.MetricStore(store_backend=None)

Bases: great_expectations.data_context.store.store.Store

A MetricStore stores ValidationMetric information to be used between runs.

_key_class
_validate_value(self, value)
serialize(self, key, value)
deserialize(self, key, value)
class great_expectations.data_context.store.Store(store_backend=None, runtime_environment=None)

Bases: object

A store is responsible for reading and writing Great Expectations objects to appropriate backends. It provides a generic API that the DataContext can use independently of any particular ORM and backend.

An implementation of a store will generally need to define the following:
  • serialize

  • deserialize

  • _key_class (class of expected key type)

All keys must have a to_tuple() method.

_key_class
_validate_key(self, key)
property store_backend(self)
serialize(self, key, value)
key_to_tuple(self, key)
tuple_to_key(self, tuple_)
deserialize(self, key, value)
get(self, key)
set(self, key, value)
list_keys(self)
has_key(self, key)
class great_expectations.data_context.store.InMemoryStoreBackend(runtime_environment=None, fixed_length_key=False)

Bases: great_expectations.data_context.store.store_backend.StoreBackend

Uses an in-memory dictionary as a store backend.

_get(self, key)
_set(self, key, value, **kwargs)
_move(self, source_key, dest_key, **kwargs)
list_keys(self, prefix=())
_has_key(self, key)
remove_key(self, key)
class great_expectations.data_context.store.StoreBackend(fixed_length_key=False)

Bases: object

A store backend acts as a key-value store that can accept tuples as keys, to abstract away reading and writing to a persistence layer.

In general a StoreBackend implementation must provide implementations of:
  • _get

  • _set

  • list_keys

  • _has_key

IGNORED_FILES = ['.ipynb_checkpoints']
property fixed_length_key(self)
get(self, key)
set(self, key, value, **kwargs)
move(self, source_key, dest_key, **kwargs)
has_key(self, key)
get_url_for_key(self, key, protocol=None)
_validate_key(self, key)
_validate_value(self, value)
abstract _get(self, key)
abstract _set(self, key, value, **kwargs)
abstract _move(self, source_key, dest_key, **kwargs)
abstract list_keys(self, prefix=())
abstract remove_key(self, key)
abstract _has_key(self, key)
is_ignored_key(self, key)
class great_expectations.data_context.store.TupleFilesystemStoreBackend(base_directory, filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=True, root_directory=None, fixed_length_key=False)

Bases: great_expectations.data_context.store.tuple_store_backend.TupleStoreBackend

Uses a local filepath as a store.

The key to this StoreBackend must be a tuple with fixed length based on the filepath_template, or a variable-length tuple may be used and returned with an optional filepath_suffix (to be) added. The filepath_template is a string template used to convert the key to a filepath.

_get(self, key)
_set(self, key, value, **kwargs)
_move(self, source_key, dest_key, **kwargs)
list_keys(self, prefix=())
rrmdir(self, mroot, curpath)

recursively removes empty dirs between curpath and mroot inclusive

remove_key(self, key)
get_url_for_key(self, key, protocol=None)
_has_key(self, key)
class great_expectations.data_context.store.TupleGCSStoreBackend(bucket, project, prefix='', filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=False, fixed_length_key=False, public_urls=True)

Bases: great_expectations.data_context.store.tuple_store_backend.TupleStoreBackend

Uses a GCS bucket as a store.

The key to this StoreBackend must be a tuple with fixed length based on the filepath_template, or a variable-length tuple may be used and returned with an optional filepath_suffix (to be) added.

The filepath_template is a string template used to convert the key to a filepath.

_move(self, source_key, dest_key, **kwargs)
_get(self, key)
_set(self, key, value, content_encoding='utf-8', content_type='application/json')
list_keys(self)
get_url_for_key(self, key, protocol=None)
remove_key(self, key)
_has_key(self, key)
class great_expectations.data_context.store.TupleS3StoreBackend(bucket, prefix='', filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=False, fixed_length_key=False)

Bases: great_expectations.data_context.store.tuple_store_backend.TupleStoreBackend

Uses an S3 bucket as a store.

The key to this StoreBackend must be a tuple with fixed length based on the filepath_template, or a variable-length tuple may be used and returned with an optional filepath_suffix (to be) added. The filepath_template is a string template used to convert the key to a filepath.

_get(self, key)
_set(self, key, value, content_encoding='utf-8', content_type='application/json')
_move(self, source_key, dest_key, **kwargs)
list_keys(self)
get_url_for_key(self, key, protocol=None)
remove_key(self, key)
_has_key(self, key)
class great_expectations.data_context.store.TupleStoreBackend(filepath_template=None, filepath_prefix=None, filepath_suffix=None, forbidden_substrings=None, platform_specific_separator=True, fixed_length_key=False)

Bases: great_expectations.data_context.store.store_backend.StoreBackend

If filepath_template is provided, the key to this StoreBackend abstract class must be a tuple with fixed length equal to the number of unique components matching the regex r”{d+}”

For example, in the following template path: expectations/{0}/{1}/{2}/prefix-{2}.json, keys must have three components.

_validate_key(self, key)
_validate_value(self, value)
_convert_key_to_filepath(self, key)
_convert_filepath_to_key(self, filepath)
verify_that_key_to_filepath_operation_is_reversible(self)
class great_expectations.data_context.store.ValidationsStore(store_backend=None, runtime_environment=None)

Bases: great_expectations.data_context.store.store.Store

A store is responsible for reading and writing Great Expectations objects to appropriate backends. It provides a generic API that the DataContext can use independently of any particular ORM and backend.

An implementation of a store will generally need to define the following:
  • serialize

  • deserialize

  • _key_class (class of expected key type)

All keys must have a to_tuple() method.

_key_class
serialize(self, key, value)
deserialize(self, key, value)