DirectoryDataAsset
Base class for PathDataAssets which batch by combining the contents of a directory.
Add a BatchDefinition, which creates a single Batch for each day in the directory.
Parameters
Name Description name
Name of the Batch Definition.
column
Column to partition on.
Returns
Type Description BatchDefinition
A BatchDefinition that is partitioned daily.
Add a BatchDefinition which creates a single batch for each month in the directory.
Parameters
Name Description name
Name of the Batch Definition.
column
Column to partition on.
Returns
Type Description BatchDefinition
A BatchDefinition that is partitioned monthly.
Add a BatchDefinition which creates a single batch for the entire directory.
Add a BatchDefinition which creates a single batch for each year in the directory.
Parameters
Name Description name
Name of the Batch Definition.
column
Column to partition on.
Returns
Type Description BatchDefinition
A BatchDefinition that is partitioned yearly.
Delete a batch definition.
Parameters
Name Description name (str)
Name of the BatchDefinition to delete.
Get a batch definition.
Parameters
Name Description name (str)
Name of the BatchDefinition to get.
Raises
Type Description KeyError
If the BatchDefinition does not exist.
class great_expectations.datasource.fluent.data_asset.path.directory_asset.DirectoryDataAsset(
*,
name: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12000>,
type: <pydantic.v1.fields.DeferredType object at 0x7f23e9c120c0>,
id: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12180> = None,
order_by: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12240> = None,
batch_metadata: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12390> = None,
batch_definitions: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12540> = None,
connect_options: <pydantic.v1.fields.DeferredType object at 0x7f23e9c12600> = None,
data_directory: pathlib.Path,
**extra_data: typing.Any
)
Methods
add_batch_definition_daily
add_batch_definition_daily(
name: str,
column: str
) → BatchDefinition
add_batch_definition_monthly
add_batch_definition_monthly(
name: str,
column: str
) → BatchDefinition
add_batch_definition_whole_directory
add_batch_definition_whole_directory(
name: str
) → BatchDefinition
add_batch_definition_yearly
add_batch_definition_yearly(
name: str,
column: str
) → BatchDefinition
delete_batch_definition
delete_batch_definition(
name: str
) → None
get_batch_definition
get_batch_definition(
name: str
) → great_expectations.core.batch_definition.BatchDefinition