Command Line Interface (CLI) Module

class great_expectations.cli.datasource.DatasourceTypes

Bases: enum.Enum

An enumeration.

PANDAS = 'pandas'
SQL = 'sql'
SPARK = 'spark'
class great_expectations.cli.datasource.SupportedDatabases

Bases: enum.Enum

An enumeration.

MYSQL = 'MySQL'
POSTGRES = 'Postgres'
REDSHIFT = 'Redshift'
SNOWFLAKE = 'Snowflake'
BIGQUERY = 'BigQuery'
OTHER = 'other - Do you have a working SQLAlchemy connection string?'
great_expectations.cli.datasource.add_datasource(context, choose_one_data_asset=False)

Interactive flow for adding a datasource to an existing context.

Parameters
  • context

  • choose_one_data_asset – optional - if True, this signals the method that the intent is to let user choose just one data asset (e.g., a file) and there is no need to configure a batch kwargs generator that comprehensively scans the datasource for data assets

Returns

a tuple: datasource_name, data_source_type

great_expectations.cli.datasource.load_library(library_name, install_instructions_string=None)

Dynamically load a module from strings or raise a helpful error.

Parameters
  • library_name – name of the library to load

  • install_instructions_string – optional - used when the install instructions are different from ‘pip install library_name’

Returns

True if the library was loaded successfully, False otherwise

great_expectations.cli.datasource.select_batch_kwargs_generator(context, datasource_name, available_data_assets_dict=None)
great_expectations.cli.datasource.get_batch_kwargs(context, datasource_name=None, batch_kwargs_generator_name=None, data_asset_name=None, additional_batch_kwargs=None)

This method manages the interaction with user necessary to obtain batch_kwargs for a batch of a data asset.

In order to get batch_kwargs this method needs datasource_name, batch_kwargs_generator_name and data_asset_name to combine them into a fully qualified data asset identifier(datasource_name/batch_kwargs_generator_name/data_asset_name). All three arguments are optional. If they are present, the method uses their values. Otherwise, the method prompts user to enter them interactively. Since it is possible for any of these three components to be passed to this method as empty values and to get their values after interacting with user, this method returns these components’ values in case they changed.

If the datasource has batch_kwargs_generators that can list available data asset names, the method lets user choose a name from that list (note: if there are multiple batch_kwargs_generators, user has to choose one first). If a name known to the chosen batch_kwargs_generator is selected, the batch_kwargs_generators will be able to yield batch_kwargs. The method also gives user an alternative to selecting the data asset name from the batch_kwargs_generators’s list - user can type in a name for their data asset. In this case a passthrough batch kwargs batch_kwargs_generators will be used to construct a fully qualified data asset identifier (note: if the datasource has no passthrough batch_kwargs_generators configured, the method will exist with a failure). Since no batch_kwargs_generators can yield batch_kwargs for this data asset name, the method prompts user to specify batch_kwargs by choosing a file (if the datasource is pandas or spark) or by writing a SQL query (if the datasource points to a database).

Parameters
  • context

  • datasource_name

  • batch_kwargs_generator_name

  • data_asset_name

  • additional_batch_kwargs

Returns

a tuple: (datasource_name, batch_kwargs_generator_name, data_asset_name, batch_kwargs). The components of the tuple were passed into the methods as optional arguments, but their values might have changed after this method’s execution. If the returned batch_kwargs is None, it means that the batch_kwargs_generator will know to yield batch_kwargs when called.

great_expectations.cli.datasource.profile_datasource(context, datasource_name, batch_kwargs_generator_name=None, data_assets=None, profile_all_data_assets=False, max_data_assets=20, additional_batch_kwargs=None, open_docs=False)

“Profile a named datasource using the specified context

great_expectations.cli.project.do_config_check(target_directory)
great_expectations.cli.docs.build_docs(context, site_name=None, view=True)

Build documentation in a context

great_expectations.cli.cli.main()
class great_expectations.cli.toolkit.MyYAML(_kw: Any = <object object>, typ: Optional[Text] = None, pure: Any = False, output: Any = None, plug_ins: Any = None)

Bases: ruamel.yaml.main.YAML

dump(data, stream=None, **kw)
great_expectations.cli.toolkit.create_expectation_suite(context, datasource_name=None, batch_kwargs_generator_name=None, generator_asset=None, batch_kwargs=None, expectation_suite_name=None, additional_batch_kwargs=None, empty_suite=False, show_intro_message=False, flag_build_docs=True, open_docs=False, profiler_configuration='demo', data_asset_name=None)

Create a new expectation suite.

WARNING: the flow and name of this method and its interaction with _profile_to_create_a_suite require a serious revisiting. :return: a tuple: (success, suite name, profiling_results)

great_expectations.cli.toolkit.attempt_to_open_validation_results_in_data_docs(context, profiling_results)
great_expectations.cli.toolkit.tell_user_suite_exists(suite_name: str) → None
great_expectations.cli.toolkit.create_empty_suite(context: great_expectations.data_context.data_context.DataContext, expectation_suite_name: str, batch_kwargs) → None
great_expectations.cli.toolkit.launch_jupyter_notebook(notebook_path: str) → None
great_expectations.cli.toolkit.load_batch(context: great_expectations.data_context.data_context.DataContext, suite: Union[str, great_expectations.core.ExpectationSuite], batch_kwargs: Union[dict, great_expectations.core.id_dict.BatchKwargs])great_expectations.data_asset.data_asset.DataAsset
great_expectations.cli.toolkit.load_expectation_suite(context: great_expectations.data_context.data_context.DataContext, suite_name: str, usage_event: str) → great_expectations.core.ExpectationSuite

Load an expectation suite from a given context.

Handles a suite name with or without .json :param usage_event:

great_expectations.cli.toolkit.exit_with_failure_message_and_stats(context: great_expectations.data_context.data_context.DataContext, usage_event: str, message: str) → None
great_expectations.cli.toolkit.load_checkpoint(context: great_expectations.data_context.data_context.DataContext, checkpoint_name: str, usage_event: str) → dict

Load a checkpoint or raise helpful errors.

great_expectations.cli.toolkit.select_datasource(context: great_expectations.data_context.data_context.DataContext, datasource_name: str = None) → great_expectations.datasource.datasource.Datasource

Select a datasource interactively.

great_expectations.cli.toolkit.load_data_context_with_error_handling(directory: str, from_cli_upgrade_command: bool = False) → great_expectations.data_context.data_context.DataContext

Return a DataContext with good error handling and exit codes.

great_expectations.cli.toolkit.upgrade_project(context_root_dir, ge_config_version, from_cli_upgrade_command=False)
great_expectations.cli.toolkit.confirm_proceed_or_exit(confirm_prompt='Would you like to proceed?', continuation_message='Ok, exiting now. You can always read more at https://docs.greatexpectations.io/ !', exit_on_no=True)

Every CLI command that starts a potentially lengthy (>1 sec) computation or modifies some resources (e.g., edits the config file, adds objects to the stores) must follow this pattern: 1. Explain which resources will be created/modified/deleted 2. Use this method to ask for user’s confirmation

The goal of this standardization is for the users to expect consistency - if you saw one command, you know what to expect from all others.

If the user does not confirm, the program should exit. The purpose of the exit_on_no parameter is to provide the option to perform cleanup actions before exiting outside of the function.

great_expectations.cli.util.cli_message(string)
great_expectations.cli.util.cli_colorize_string(string)
great_expectations.cli.util.cli_message_list(string_list, list_intro_string=None)

Simple util function for displaying simple lists in cli

great_expectations.cli.util.action_list_to_string(action_list)

Util function for turning an action list into pretty string

great_expectations.cli.util.cli_message_dict(dict_, indent=3, bullet_char='-', message_list=None, recursion_flag=False)

Util function for displaying nested dicts representing ge objects in cli

great_expectations.cli.util.is_sane_slack_webhook(url)

Really basic sanity checking.

class great_expectations.cli.mark.Mark

Bases: object

Marks for feature readiness.

Usage: from great_expectations.cli.mark import Mark as mark

@mark.blah def your_function()

static cli_as_experimental(func: Callable) → Callable

Apply as a decorator to CLI commands that are Experimental.

static cli_as_beta(func: Callable) → Callable

Apply as a decorator to CLI commands that are beta.

static cli_as_deprecation(message: str = '<yellow>Heads up! This feature will be deprecated in the next major release</yellow>') → Callable

Apply as a decorator to CLI commands that will be deprecated.