great_expectations.self_check.util

Module Contents

Classes

SqlAlchemyConnectionManager()

LockingConnectionCheck(sa, connection_string)

Functions

get_dataset(dataset_type, data, schemas=None, profiler=ColumnsExistProfiler, caching=True, table_name=None, sqlite_db_path=None)

Utility to create datasets for json-formatted tests.

get_test_validator_with_data(execution_engine, data, schemas=None, profiler=ColumnsExistProfiler, caching=True, table_name=None, sqlite_db_path=None)

Utility to create datasets for json-formatted tests.

build_pandas_validator_with_data(df: pd.DataFrame)

build_sa_validator_with_data(df, sa_engine_name, schemas=None, caching=True, table_name=None, sqlite_db_path=None)

modify_locale(func)

build_spark_validator_with_data(df: Union[pd.DataFrame, SparkDataFrame], spark: SparkSession)

build_pandas_engine(df: pd.DataFrame)

build_sa_engine(df: pd.DataFrame, sa: ModuleType, schema: Optional[str] = None, if_exists: Optional[str] = ‘fail’, index: Optional[bool] = False, dtype: Optional[dict] = None)

build_spark_engine(spark: SparkSession, df: Union[pd.DataFrame, SparkDataFrame], batch_id: str)

candidate_getter_is_on_temporary_notimplemented_list(context, getter)

candidate_test_is_on_temporary_notimplemented_list(context, expectation_type)

candidate_test_is_on_temporary_notimplemented_list_cfe(context, expectation_type)

build_test_backends_list(include_pandas=True, include_spark=True, include_sqlalchemy=True, include_postgresql=False, include_mysql=False, include_mssql=False)

generate_expectation_tests(expectation_type, examples_config, expectation_execution_engines_dict=None)

param expectation_type

snake_case name of the expectation type

evaluate_json_test(data_asset, expectation_type, test)

This method will evaluate the result of a test build using the Great Expectations json test format.

evaluate_json_test_cfe(validator, expectation_type, test)

This method will evaluate the result of a test build using the Great Expectations json test format.

check_json_test_result(test, result, data_asset=None)

generate_test_table_name(default_table_name_prefix: Optional[str] = ‘test_data_’)

great_expectations.self_check.util.expectationValidationResultSchema
great_expectations.self_check.util.expectationSuiteValidationResultSchema
great_expectations.self_check.util.expectationConfigurationSchema
great_expectations.self_check.util.expectationSuiteSchema
great_expectations.self_check.util.logger
great_expectations.self_check.util.tmp_dir
great_expectations.self_check.util.sqlalchemy
great_expectations.self_check.util.SparkSession
great_expectations.self_check.util.spark_DataFrame
great_expectations.self_check.util.SQLITE_TYPES
great_expectations.self_check.util.POSTGRESQL_TYPES
great_expectations.self_check.util.MYSQL_TYPES
great_expectations.self_check.util.MSSQL_TYPES
class great_expectations.self_check.util.SqlAlchemyConnectionManager
get_engine(self, connection_string)
great_expectations.self_check.util.connection_manager
class great_expectations.self_check.util.LockingConnectionCheck(sa, connection_string)
is_valid(self)
great_expectations.self_check.util.get_dataset(dataset_type, data, schemas=None, profiler=ColumnsExistProfiler, caching=True, table_name=None, sqlite_db_path=None)

Utility to create datasets for json-formatted tests.

great_expectations.self_check.util.get_test_validator_with_data(execution_engine, data, schemas=None, profiler=ColumnsExistProfiler, caching=True, table_name=None, sqlite_db_path=None)

Utility to create datasets for json-formatted tests.

great_expectations.self_check.util.build_pandas_validator_with_data(df: pd.DataFrame) → Validator
great_expectations.self_check.util.build_sa_validator_with_data(df, sa_engine_name, schemas=None, caching=True, table_name=None, sqlite_db_path=None)
great_expectations.self_check.util.modify_locale(func)
great_expectations.self_check.util.build_spark_validator_with_data(df: Union[pd.DataFrame, SparkDataFrame], spark: SparkSession) → Validator
great_expectations.self_check.util.build_pandas_engine(df: pd.DataFrame) → PandasExecutionEngine
great_expectations.self_check.util.build_sa_engine(df: pd.DataFrame, sa: ModuleType, schema: Optional[str] = None, if_exists: Optional[str] = 'fail', index: Optional[bool] = False, dtype: Optional[dict] = None) → SqlAlchemyExecutionEngine
great_expectations.self_check.util.build_spark_engine(spark: SparkSession, df: Union[pd.DataFrame, SparkDataFrame], batch_id: str) → SparkDFExecutionEngine
great_expectations.self_check.util.candidate_getter_is_on_temporary_notimplemented_list(context, getter)
great_expectations.self_check.util.candidate_test_is_on_temporary_notimplemented_list(context, expectation_type)
great_expectations.self_check.util.candidate_test_is_on_temporary_notimplemented_list_cfe(context, expectation_type)
great_expectations.self_check.util.build_test_backends_list(include_pandas=True, include_spark=True, include_sqlalchemy=True, include_postgresql=False, include_mysql=False, include_mssql=False)
great_expectations.self_check.util.generate_expectation_tests(expectation_type, examples_config, expectation_execution_engines_dict=None)
Parameters
  • expectation_type – snake_case name of the expectation type

  • examples_config – a dictionary that defines the data and test cases for the expectation

  • expectation_execution_engines_dict

    (optional) a dictionary that shows which backends/execution engines the expectation is implemented for. It can be obtained from the output of the expectation’s self_check method Example: {

    ”PandasExecutionEngine”: True, “SqlAlchemyExecutionEngine”: False, “SparkDFExecutionEngine”: False

    }

Returns

great_expectations.self_check.util.evaluate_json_test(data_asset, expectation_type, test)

This method will evaluate the result of a test build using the Great Expectations json test format.

NOTE: Tests can be suppressed for certain data types if the test contains the Key ‘suppress_test_for’ with a list

of DataAsset types to suppress, such as [‘SQLAlchemy’, ‘Pandas’].

Parameters
  • data_asset – (DataAsset) A great expectations DataAsset

  • expectation_type – (string) the name of the expectation to be run using the test input

  • test

    (dict) a dictionary containing information for the test to be run. The dictionary must include: - title: (string) the name of the test - exact_match_out: (boolean) If true, match the ‘out’ dictionary exactly against the result of the expectation - in: (dict or list) a dictionary of keyword arguments to use to evaluate the expectation or a list of positional arguments - out: (dict) the dictionary keys against which to make assertions. Unless exact_match_out is true, keys must come from the following list:

    • success

    • observed_value

    • unexpected_index_list

    • unexpected_list

    • details

    • traceback_substring (if present, the string value will be expected as a substring of the exception_traceback)

Returns

None. asserts correctness of results.

great_expectations.self_check.util.evaluate_json_test_cfe(validator, expectation_type, test)

This method will evaluate the result of a test build using the Great Expectations json test format.

NOTE: Tests can be suppressed for certain data types if the test contains the Key ‘suppress_test_for’ with a list

of DataAsset types to suppress, such as [‘SQLAlchemy’, ‘Pandas’].

Parameters
  • data_asset – (DataAsset) A great expectations DataAsset

  • expectation_type – (string) the name of the expectation to be run using the test input

  • test

    (dict) a dictionary containing information for the test to be run. The dictionary must include: - title: (string) the name of the test - exact_match_out: (boolean) If true, match the ‘out’ dictionary exactly against the result of the expectation - in: (dict or list) a dictionary of keyword arguments to use to evaluate the expectation or a list of positional arguments - out: (dict) the dictionary keys against which to make assertions. Unless exact_match_out is true, keys must come from the following list:

    • success

    • observed_value

    • unexpected_index_list

    • unexpected_list

    • details

    • traceback_substring (if present, the string value will be expected as a substring of the exception_traceback)

Returns

None. asserts correctness of results.

great_expectations.self_check.util.check_json_test_result(test, result, data_asset=None)
great_expectations.self_check.util.generate_test_table_name(default_table_name_prefix: Optional[str] = 'test_data_') → str