Skip to main content
Version: 1.3.7

DataFrameAsset

Signature

class great_expectations.datasource.fluent.spark_datasource.DataFrameAsset(*, name: str, type: Literal['dataframe'] = 'dataframe', id: Optional[uuid.UUID] = None, order_by: List[great_expectations.datasource.fluent.interfaces.Sorter] = None, batch_metadata: Dict[str, Any] = None, batch_definitions: List[great_expectations.core.batch_definition.BatchDefinition] = None)

A DataAsset that represents a Spark DataFrame.

Methods

Signature

add_batch_definition_whole_dataframe(name: str) → BatchDefinition

Add a BatchDefinition that represents the entire DataFrame.

Parameters

NameDescription

name

The name of the Batch Definition.

Returns

TypeDescription

BatchDefinition

A BatchDefinition object that represents the entire DataFrame.

Signature

delete_batch_definition(name: str)None

Delete a batch definition.

Parameters

NameDescription

name (str)

Name of the BatchDefinition to delete.

Signature

get_batch_definition(name: str) → great_expectations.core.batch_definition.BatchDefinition

Get a batch definition.

Parameters

NameDescription

name (str)

Name of the BatchDefinition to get.

Raises

TypeDescription

KeyError

If the BatchDefinition does not exist.