Skip to main content
Version: 1.3.0

DatabricksSQLDatasource

class great_expectations.datasource.fluent.DatabricksSQLDatasource(*, type: Literal['databricks_sql'] = 'databricks_sql', name: str, id: Optional[uuid.UUID] = None, assets: List[Union[great_expectations.datasource.fluent.sql_datasource.TableAsset, great_expectations.datasource.fluent.sql_datasource.QueryAsset]] = [], connection_string: Union[great_expectations.datasource.fluent.config_str.ConfigStr, great_expectations.datasource.fluent.databricks_sql_datasource.DatabricksDsn], create_temp_table: bool = False, kwargs: Dict[str, Union[great_expectations.datasource.fluent.config_str.ConfigStr, Any]] = )#

Adds a DatabricksSQLDatasource to the data context.

Parameters

  • name – The name of this DatabricksSQL datasource.

  • connection_string – The SQLAlchemy connection string used to connect to the Databricks SQL database. For example: “databricks://token:<token>@<host>:<port>?http_path=<http_path>&catalog=<catalog>&schema=<schema>””

  • assets – An optional dictionary whose keys are TableAsset or QueryAsset names and whose values are TableAsset or QueryAsset objects.

add_query_asset(name: str, query: str, batch_metadata: Optional[BatchMetadata] = None) → QueryAsset#

Adds a query asset to this datasource.

Parameters

  • name – The name of this table asset.

  • query – The SELECT query to selects the data to validate. It must begin with the “SELECT”.

  • batch_metadata – BatchMetadata we want to associate with this DataAsset and all batches derived from it.

Returns

The query asset that is added to the datasource. The type of this object will match the necessary type for this datasource. eg, it could be a QueryAsset or a SqliteQueryAsset.

add_table_asset(name: str, table_name: str = '', schema_name: Optional[str] = None, batch_metadata: Optional[BatchMetadata] = None) → TableAsset#

Adds a table asset to this datasource.

Parameters

  • name – The name of this table asset.

  • table_name – The table where the data resides.

  • schema_name – The schema that holds the table.

  • batch_metadata – BatchMetadata we want to associate with this DataAsset and all batches derived from it.

Returns

The table asset that is added to the datasource. The type of this object will match the necessary type for this datasource. eg, it could be a TableAsset or a SqliteTableAsset.

delete_asset(name: str)→ None#

Removes the DataAsset referred to by asset_name from internal list of available DataAsset objects.

Parameters

name – name of DataAsset to be deleted.

get_asset(name: str)→ great_expectations.datasource.fluent.interfaces._DataAssetT#

Returns the DataAsset referred to by asset_name

Parameters

name – name of DataAsset sought.

Returns

_DataAssetT – if named “DataAsset” object exists; otherwise, exception is raised.