FileDataAsset
Base class for PathDataAssets which batch by applying a regex to file names.
Add a BatchDefinition which defines daily batches by file name.
Parameters
Name Description name
BatchDefinition name
regex
Regular Expression used to define batches by file name. Must contain the groups year, month, and day.
sort_ascending
determine order in which batches are returned
Raises
Type Description RegexMissingRequiredGroupsError
regex is missing the groups year, month, and/or day.
RegexUnknownGroupsError
regex has groups other than year, month, and/or day.
Add a BatchDefinition which defines monthly batches by file name.
Parameters
Name Description name
BatchDefinition name
regex
Regular Expression used to define batches by file name. Must contain the groups year and month.
sort_ascending
determine order in which batches are returned
Raises
Type Description RegexMissingRequiredGroupsError
regex is missing the groups year and/or month.
RegexUnknownGroupsError
regex has groups other than year and/or month.
Add a BatchDefinition which matches a single Path.
Parameters
Name Description name
BatchDefinition name
path
File path relative to the Asset
Raises
Type Description PathNotFoundError
path cannot be resolved
AmbiguousPathError
path matches more than one file
Add a BatchDefinition which defines yearly batches by file name.
Parameters
Name Description name
BatchDefinition name
regex
Regular Expression used to define batches by file name. Must contain a single group year
sort_ascending
determine order in which batches are returned
Raises
Type Description RegexMissingRequiredGroupsError
regex is missing the group year
RegexUnknownGroupsError
regex has groups other than year
Delete a batch definition.
Parameters
Name Description name (str)
Name of the BatchDefinition to delete.
Get a batch definition.
Parameters
Name Description name (str)
Name of the BatchDefinition to get.
Raises
Type Description KeyError
If the BatchDefinition does not exist.
class great_expectations.datasource.fluent.data_asset.path.file_asset.FileDataAsset(
*,
name: <pydantic.v1.fields.DeferredType object at 0x7fa559a48f20>,
type: <pydantic.v1.fields.DeferredType object at 0x7fa559a48fe0>,
id: <pydantic.v1.fields.DeferredType object at 0x7fa559a490a0> = None,
order_by: <pydantic.v1.fields.DeferredType object at 0x7fa559a49160> = None,
batch_metadata: <pydantic.v1.fields.DeferredType object at 0x7fa559a492b0> = None,
batch_definitions: <pydantic.v1.fields.DeferredType object at 0x7fa559a49460> = None,
connect_options: <pydantic.v1.fields.DeferredType object at 0x7fa559a49520> = None,
**extra_data: typing.Any
)
Methods
add_batch_definition_daily
add_batch_definition_daily(
name: str,
regex: Union[re.Pattern,
str],
sort_ascending: bool = True
) → BatchDefinition
add_batch_definition_monthly
add_batch_definition_monthly(
name: str,
regex: Union[re.Pattern,
str],
sort_ascending: bool = True
) → BatchDefinition
add_batch_definition_path
add_batch_definition_path(
name: str,
path: PathStr
) → BatchDefinition
add_batch_definition_yearly
add_batch_definition_yearly(
name: str,
regex: Union[re.Pattern,
str],
sort_ascending: bool = True
) → BatchDefinition
delete_batch_definition
delete_batch_definition(
name: str
) → None
get_batch_definition
get_batch_definition(
name: str
) → great_expectations.core.batch_definition.BatchDefinition