great_expectations.marshmallow__shade.schema

The Schema class, including its metaclass and options (class Meta).

Module Contents

Classes

SchemaMeta(cls, name, bases, attrs)

Metaclass for the Schema class. Binds the declared fields to

SchemaOpts(meta, ordered: bool = False)

class Meta options for the Schema. Defines defaults.

Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

Functions

_get_fields(attrs, field_class, pop=False, ordered=False)

Get fields from a class. If ordered=True, fields will sorted by creation index.

_get_fields_by_mro(klass, field_class, ordered=False)

Collect fields from a class, following its method resolution order. The

great_expectations.marshmallow__shade.schema._T
great_expectations.marshmallow__shade.schema._get_fields(attrs, field_class, pop=False, ordered=False)

Get fields from a class. If ordered=True, fields will sorted by creation index.

Parameters
  • attrs – Mapping of class attributes

  • field_class (type) – Base field class

  • pop (bool) – Remove matching fields

great_expectations.marshmallow__shade.schema._get_fields_by_mro(klass, field_class, ordered=False)

Collect fields from a class, following its method resolution order. The class itself is excluded from the search; only its parents are checked. Get fields from _declared_fields if available, else use __dict__.

Parameters
  • klass (type) – Class whose fields to retrieve

  • field_class (type) – Base field class

class great_expectations.marshmallow__shade.schema.SchemaMeta(cls, name, bases, attrs)

Bases: type

Metaclass for the Schema class. Binds the declared fields to a _declared_fields attribute, which is a dictionary mapping attribute names to field objects. Also sets the opts class attribute, which is the Schema class’s class Meta options.

classmethod get_declared_fields(mcs, klass: type, cls_fields: typing.List, inherited_fields: typing.List, dict_cls: type)

Returns a dictionary of field_name => Field pairs declared on the class. This is exposed mainly so that plugins can add additional fields, e.g. fields computed from class Meta options.

Parameters
  • klass – The class object.

  • cls_fields – The fields declared on the class, including those added by the include class Meta option.

  • inherited_fields – Inherited fields.

  • dict_class – Either dict or OrderedDict, depending on the whether the user specified ordered=True.

resolve_hooks(cls)

Add in the decorated processors

By doing this after constructing the class, we let standard inheritance do all the hard work.

class great_expectations.marshmallow__shade.schema.SchemaOpts(meta, ordered: bool = False)

class Meta options for the Schema. Defines defaults.

class great_expectations.marshmallow__shade.schema.Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.base.SchemaABC

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

class Meta

Options object for a Schema.

Example usage:

class Meta:
    fields = ("id", "email", "date_created")
    exclude = ("password", "secret_attribute")

Available options:

  • fields: Tuple or list of fields to include in the serialized result.

  • additional: Tuple or list of fields to include in addition to the

    explicitly declared fields. additional and fields are mutually-exclusive options.

  • include: Dictionary of additional fields to include in the schema. It is

    usually better to define fields as class variables, but you may need to use this option, e.g., if your fields are Python keywords. May be an OrderedDict.

  • exclude: Tuple or list of fields to exclude in the serialized result.

    Nested fields can be represented with dot delimiters.

  • dateformat: Default format for Date <fields.Date> fields.

  • datetimeformat: Default format for DateTime <fields.DateTime> fields.

  • render_module: Module to use for loads <Schema.loads> and dumps <Schema.dumps>.

    Defaults to json from the standard library.

  • ordered: If True, order serialization output according to the

    order in which fields were declared. Output of Schema.dump will be a collections.OrderedDict.

  • index_errors: If True, errors dictionaries will include the index

    of invalid items in a collection.

  • load_only: Tuple or list of fields to exclude from serialized results.

  • dump_only: Tuple or list of fields to exclude from deserialization

  • unknown: Whether to exclude, include, or raise an error for unknown

    fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • register: Whether to register the Schema with marshmallow’s internal

    class registry. Must be True if you intend to refer to this Schema by class name in Nested fields. Only set this to False when memory usage is critical. Defaults to True.

TYPE_MAPPING :typing.Dict[type, typing.Type[ma_fields.Field]]
error_messages :typing.Dict[str, str]
_default_error_messages :typing.Dict[str, str]
OPTIONS_CLASS :type
opts :SchemaOpts
_declared_fields :typing.Dict[str, ma_fields.Field]
_hooks :typing.Dict[types.Tag, typing.List[str]]
__repr__(self)

Return repr(self).

property dict_class(self)
property set_class(self)
classmethod from_dict(cls, fields: typing.Dict[str, typing.Union[ma_fields.Field, type]], *, name: str = 'GeneratedSchema')

Generate a Schema class given a dictionary of fields.

from great_expectations.marshmallow__shade import Schema, fields

PersonSchema = Schema.from_dict({"name": fields.Str()})
print(PersonSchema().load({"name": "David"}))  # => {'name': 'David'}

Generated schemas are not added to the class registry and therefore cannot be referred to by name in Nested fields.

Parameters
  • fields (dict) – Dictionary mapping field names to field instances.

  • name (str) – Optional name for the class, which will appear in the repr for the class.

New in version 3.0.0.

handle_error(self, error: ValidationError, data: typing.Any, *, many: bool, **kwargs)

Custom error handler function for the schema.

Parameters
  • error – The ValidationError raised during (de)serialization.

  • data – The original input data.

  • many – Value of many on dump or load.

  • partial – Value of partial on load.

New in version 2.0.0.

Changed in version 3.0.0rc9: Receives many and partial (on deserialization) as keyword arguments.

get_attribute(self, obj: typing.Any, attr: str, default: typing.Any)

Defines how to pull values from an object to serialize.

New in version 2.0.0.

Changed in version 3.0.0a1: Changed position of obj and attr.

static _call_and_store(getter_func, data, *, field_name, error_store, index=None)

Call getter_func with data as its argument, and store any ValidationErrors.

Parameters
  • getter_func (callable) – Function for getting the serialized/deserialized value from data.

  • data – The data passed to getter_func.

  • field_name (str) – Field name.

  • index (int) – Index of the item being validated, if validating a collection, otherwise None.

_serialize(self, obj: typing.Union[_T, typing.Iterable[_T]], *, many: bool = False)

Serialize obj.

Parameters
  • obj – The object(s) to serialize.

  • many (bool) – True if data should be serialized as a collection.

Returns

A dictionary of the serialized data

Changed in version 1.0.0: Renamed from marshal.

dump(self, obj: typing.Any, *, many: bool = None)

Serialize an object to native Python data types according to this Schema’s fields.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A dict of serialized data

Return type

dict

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

Changed in version 3.0.0rc9: Validation no longer occurs upon serialization.

dumps(self, obj: typing.Any, *args, many: bool = None, **kwargs)

Same as dump(), except return a JSON-encoded string.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A json string

Return type

str

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

_deserialize(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, error_store: ErrorStore, many: bool = False, partial=False, unknown=RAISE, index=None)

Deserialize data.

Parameters
  • data (dict) – The data to deserialize.

  • error_store (ErrorStore) – Structure to store errors.

  • many (bool) – True if data should be deserialized as a collection.

  • partial (bool|tuple) – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • index (int) – Index of the item being serialized (for storing errors) if serializing a collection, otherwise None.

Returns

A dictionary of the deserialized data.

load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None)

Deserialize a data structure to an object defined by this Schema’s fields.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

loads(self, json_data: str, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, **kwargs)

Same as load(), except it takes a JSON string as input.

Parameters
  • json_data – A JSON string of the data to deserialize.

  • many – Whether to deserialize obj as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

_run_validator(self, validator_func, output, *, original_data, error_store, many, partial, pass_original, index=None)
validate(self, data: typing.Mapping, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None)

Validate data against the schema, returning a dictionary of validation errors.

Parameters
  • data – The data to validate.

  • many – Whether to validate data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

Returns

A dictionary of validation errors.

New in version 1.1.0.

_do_load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, postprocess: bool = True)

Deserialize data, returning the deserialized result. This method is private API.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to validate required fields. If its value is an iterable, only fields listed in that iterable will be ignored will be allowed missing. If True, all fields will be allowed missing. If None, the value for self.partial is used.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

  • postprocess – Whether to run post_load methods..

Returns

Deserialized data

_normalize_nested_options(self)

Apply then flatten nested schema options. This method is private API.

__apply_nested_option(self, option_name, field_names, set_operation)

Apply nested options to nested fields

_init_fields(self)

Update self.fields, self.load_fields, and self.dump_fields based on schema options. This method is private API.

on_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Hook to modify a field when it is bound to the Schema.

No-op by default.

_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Bind field to the schema, setting any necessary attributes on the field (e.g. parent and name).

Also set field load_only and dump_only values if field_name was specified in class Meta.

_has_processors(self, tag)
_invoke_dump_processors(self, tag: str, data, *, many: bool, original_data=None)
_invoke_load_processors(self, tag: str, data, *, many: bool, original_data, partial: typing.Union[bool, types.StrSequenceOrSet])
_invoke_field_validators(self, *, error_store: ErrorStore, data, many: bool)
_invoke_schema_validators(self, *, error_store: ErrorStore, pass_many: bool, data, original_data, many: bool, partial: typing.Union[bool, types.StrSequenceOrSet], field_errors: bool = False)
_invoke_processors(self, tag: str, *, pass_many: bool, data, many: bool, original_data=None, **kwargs)
great_expectations.marshmallow__shade.schema.BaseSchema