Changelog
Deprecation policy
GX Core follows Semantic Versioning 2.0.0, including its guidelines for deprecation.
When we deprecate public functionality, we will
- update our documentation to let you know about the change.
- issue a new minor release with the deprecation in place.
Before we completely remove the functionality in a new major release, there will be at least one minor release that contains the deprecation so that you can smoothly transition.
1.5.3
- [BUGFIX] Make ExpectTableColumnsToMatchSet case insensitive (#11266)
- [DOCS] ExpectColumnProportionOfNonNullValuesToBeBetween (#11245) (#11257)
- [MAINTENANCE] Allow posthog v4 and v5 (#11265)
1.5.2
- [FEATURE] Add SuiteParameterDict to all Expectation Kwargs (#11222) (thanks @Pascal06S)
- [BUGFIX] fix
ValidationError
inExpectColumnPairValuesToHaveDifferenceOfCustomPercentage
(#11209) (thanks @sariaslaso) - [BUGFIX]
add_batch_definition_whole_directory
only loading 1 file (#11254) - [BUGFIX] Remove datetime types from proportion
min_value
andmax_value
parameters (#11259) - [DOCS] temporarily remove link checker (#11250)
- [DOCS] Cloud API data source tip (#11248)
- [DOCS] ExpectColumnProportionOfNonNullValuesToBeBetween (#11245) (#11257)
- [MAINTENANCE] Update "fraction" to "proportion" on NonNullProportion renderer (#11253)
- [MAINTENANCE] Bump brace-expansion from 1.1.11 to 1.1.12 in /docs/docusaurus (#11251)
1.5.1
- [FEATURE] Add ColumnAggregateNonNullCount metric (#11229)
- [FEATURE]
ExpectColumnProportionOfUniqueValuesToBeBetween
(#11235) - [DOCS] Revise use of the term "API" (#11196)
- [DOCS] fix link to removed content (#11232)
- [MAINTENANCE] Revert "[MAINTENANCE] Snowflake tests only run with snowflake flag" (#11230)
- [MAINTENANCE] Add session pool to expectation tests (#11228)
- [MAINTENANCE] Remove RuleBasedProfiler (#11231)
1.5.0
- [MINORBUMP] docs for Multi-source Expectations (#11165)
- [DOCS] revise use of the term "Cloud" in the Core docs (#11207)
- [DOCS] restore link checker (#11212)
- [MAINTENANCE] add warning filter (#11217)
- [MAINTENANCE] Update renderer labels (#11216)
- [MAINTENANCE] Improve
ExpectQueryResultsToMatchComparison
docstring (#11221) - [MAINTENANCE] pre-commit autoupdate (#11218)
- [MAINTENANCE] Remove
pkg_resources
dependency (#11213) - [MAINTENANCE] Example suite parameter tests (#11225)
- [CONTRIB] Add type geometry and super into redshift_execution_engine.py (#11194) (thanks @VolkovGeoPhy)
1.4.6
- [BUGFIX] Handle unhashable data types for ExpectQueryResultsToMatchComparison (#11193)
- [BUGFIX] Make type expectation case-insensitive for Databricks, Snowflake an Postgres (#11192)
- [BUGFIX] Handle set rendering cases for
ExpectQueryResultsToMatchComparison
(#11203) - [BUGFIX] Handle UUID in rendered content (#11204)
- [DOCS] Anomaly Detection (#11172)
- [DOCS] Refine use of the term "platform" (#11187)
- [DOCS] Update docs to reflect that views are supported (#11198)
- [DOCS] Revise use of the term "Core" in relation to Cloud (#11205)
- [MAINTENANCE] Add CloudAMQP connection string env variable to ci (#11190)
- [MAINTENANCE] Show gx cloud logs on cloud test failures. (#11188)
- [MAINTENANCE] limit pyspark to <4.0 due to breaking changes in types (#11197)
- [MAINTENANCE] Ignore warnings around pkg_resources (#11201)
- [MAINTENANCE] Update descriptions for multi-source parameter (#11202)
- [MAINTENANCE] Snowflake tests only run with snowflake flag (#10605)
1.4.5
- [DOCS] ExpectAI for all Data Sources (#11178)
- [MAINTENANCE] Fix nightly data source cleanup action setup (#11176)
- [MAINTENANCE] pre-commit autoupdate (#10480)
- [MAINTENANCE] Do not fail tests on snowflake ssl connection warning. (#11180)
- [MAINTENANCE] Upgrade ruff (#11182)
- [MAINTENANCE] Update Posthog analytics events (#11179)
- [MAINTENANCE] Rename multi-source Expectation and parameters (#11185)
- [MAINTENANCE] Add single value rendering path to multi-source Expectation (#11186)
- [CONTRIB] use existing LRU cache for secrets store (#11184) (thanks @ThiloSchneider-fraport)
- [CONTRIB] add GEOMETRY, and SUPER types to REDSHIFT_TYPES (#11183) (thanks @VolkovGeoPhy)
1.4.4
- [FEATURE] Compute missing/unexpected rows for ExpectQueryResultsToMatchSource (#11161)
- [FEATURE] ExpectQueryResultsToMatchSource table diagnostic renderer (#11168)
- [FEATURE] ExpectQueryResultsToMatchSource renderer for single-column … (#11173)
- [DOCS] remove link checker (#11162)
- [MAINTENANCE] Fix dying mssql compatibility tests. (#11153)
- [MAINTENANCE] Bump prismjs from 1.29.0 to 1.30.0 in /docs/docusaurus (#11126)
- [MAINTENANCE] Breakout redshift tests into own job. (#11167)
- [MAINTENANCE] Bring back redshift creds for docs tests (#11169)
- [MAINTENANCE] Improve
ExpectQueryResultsToMatchSource
docstring (#11158) - [MAINTENANCE] Nightly redshift cleanup (#11166)
- [MAINTENANCE] Bump image-size from 1.2.0 to 1.2.1 in /docs/docusaurus (#11125)
- [MAINTENANCE] Add prescriptive atomic renderer to
ExpectQueryResultsToMatchSource
(#11160) - [MAINTENANCE] Resolve datetime deprecation warnings (#11134) (thanks @emmanuel-ferdman)
- [MAINTENANCE] Bump http-proxy-middleware from 2.0.7 to 2.0.9 in /docs/docusaurus (#11120)
- [MAINTENANCE] Bump @babel/runtime from 7.26.9 to 7.27.0 in /docs/docusaurus (#11124)
- [MAINTENANCE] Bump estree-util-value-to-estree from 3.3.2 to 3.3.3 in /docs/docusaurus (#11121)
- [MAINTENANCE] ensure latest docker compose (#11156)
- [MAINTENANCE] Fix flakey test and simplify another assertion (#11163)
- [MAINTENANCE] Update
ExpectQueryResultsToMatchSource
DQI (#11164) - [MAINTENANCE] Bump @babel/helpers from 7.26.9 to 7.27.0 in /docs/docusaurus (#11123)
- [MAINTENANCE] Change DQ issue for
ExpectQueryResultsToMatchSource
(#11174) - [CONTRIB] Update ruff badge (#10905) (thanks @esadek)
1.4.3
- [FEATURE] Test infra to support source to target expectations (#11138)
- [FEATURE] QueryDataSourceTable metric + provider (#11149)
- [DOCS] ExpectAI approval workflow (#11072)
- [DOCS] restore link checker (#11135)
- [DOCS] alerts for ExpectAI (#11154)
- [MAINTENANCE] Temporarily pin gx-redshift for python 3.9 (#11142)
- [MAINTENANCE] Revert "[DOCS] restore link checker" (#11145)
- [MAINTENANCE] Capitalize "Expectation" in mostly description (#11147)
- [MAINTENANCE] Add
SupportedDataSources
enum (#11143) - [MAINTENANCE] Restore link checker (#11146)
- [MAINTENANCE] Remove slow test cases (#11152)
- [MAINTENANCE] Remove test_expectations_v3_api.py (#11098)
- [MAINTENANCE] Revert: "Temporarily pin gx-redshift for python 3.9" (#11151)
1.4.2
- [FEATURE] Changes to Redshift
connection_string
validator to support a dict type (#11119) - [FEATURE] expectations: redshift support (#11128)
- [DOCS] make try gx core code sample match descriptive text (#11129)
- [DOCS] hide broken table of contents (#11130)
- [MAINTENANCE] Add Galina to core devs. Remove others. (#11117)
- [MAINTENANCE] Add metric docstrings and fix import paths (#11118)
- [MAINTENANCE] Update type packaging (#11115)
- [MAINTENANCE] Improve
MetricErrorResult
andBatch.compute_metrics()
API typing (#11127)
1.4.1
- [FEATURE] Allow user to provide connection details to connect to Redshift (#11105)
- [FEATURE] Add
ColumnDescriptiveStats
metric (#11108) - [BUGFIX] MicrosoftTeamsNotificationAction failing with 400 Bad Request (#11106) (thanks @jwalant-dattani)
- [BUGFIX] expect_column_values_to_be_between allows both min/max values to be None or empty strings (#11102)
- [BUGFIX] redshift: prevent runtime TypeError (#11112)
- [MAINTENANCE] Cleanup metrics package (#11109)
1.4.0
- [FEATURE] Add
ColumnSampleValues
metric (#11083) - [FEATURE] Add Redshift to SUPPORTED_DATA_SOURCES where appropriate (#11084)
- [FEATURE] Add gx-redshift extra (#11092)
- [FEATURE] Add
ColumnValuesMatchRegexCount
metric (#11091) - [BUGFIX] Expectations fail validation when using sets/tuples as value_set due to JSON serialization (#11082) (thanks @gyermich)
- [DOCS] update schedule instructions (#11080)
- [DOCS] Fix default gx directory in docstring (#11077)
- [MAINTENANCE] Improve PostHog pageview tracking in docs (#11087)
- [MAINTENANCE] Add ColumnDistinctValues metric (#11081)
- [MAINTENANCE] Type narrowing on compute_metrics with a single metric (#11089)
- [MAINTENANCE] Implement ColumnValuesMatchRegexValues metric (#11088)
- [MAINTENANCE] Implement ColumnValuesNotMatchRegexValues metric (#11096)
- [MAINTENANCE] Run gx-sqlalchemy-redshift tests. (#11094)
1.3.14
- [FEATURE] Fix ExpectColumnValuesToBeOfType bug and also work with sqla2 (#11062)
- [FEATURE] Add gx-redshift extra dependency (#11063)
- [FEATURE] Enable SQLAlchemy 2.x support for BigQuery (#11059)
- [BUGFIX] Add test for 8-bit character support in row conditions parser (#11053)
- [BUGFIX] Respect context_root_dir in get_context (#11078)
- [DOCS] update schedule instructions (#11080)
- [MAINTENANCE] update docs usage of posthog (#11060)
- [MAINTENANCE] Improve package autocompletion behavior (#11070)
- [MAINTENANCE] Add BatchColumnTypes metric (#11069)
- [MAINTENANCE] Implement SampleValues metric (#11071)
- [MAINTENANCE] Implement ColumnDistinctValuesCount (#11075)
- [MAINTENANCE] Implement ColumnNullCount metric (#11073)
- [MAINTENANCE] Revert "[FEATURE] Add gx-redshift extra dependency" (#11079)
1.3.13
- [FEATURE] Add initial RedShift datasource (#11011)
- [BUGFIX] Prevent sqlite metric impl from overwriting default sqlalchemy (#11055)
- [DOCS] Add key pair authentication as a limitation of ExpectAI in manage_expectations.md (#11047) (thanks @adeola-ak)
- [DOCS] reflect change on metrics page, clarify perms (#11052)
- [DOCS] test coverage metrics part 2 (table) (#11046)
- [DOCS] completeness change detection (#11039)
- [DOCS] ccd fast follow (#11056)
- [DOCS] update coverage health screenshot (#11057)
- [MAINTENANCE] Use get_metric_provider internal to registry (#11044)
1.3.12
- [DOCS] KL divergence gallery fix (#11040)
- [MAINTENANCE] Add MINORBUMP title tag. (#10841)
- [MAINTENANCE] Add a sqlite execution engine and 1 sqlite specific metric. (#11042)
1.3.11
- [FEATURE] Add the ability to run checkpoints with cloud windowed expectations (#11027)
- [BUGFIX] Distinct values expectations correctly compare dates and datetimes (#11030)
- [DOCS] restore link checker (#11025)
- [DOCS] - single category for Expectations (#11008)
- [DOCS] fix error in filesystem data source docs (#11019)
- [MAINTENANCE] Remove default snowflake role (#11004)
- [MAINTENANCE] Add mode to props sent in analytics (#11001)
- [MAINTENANCE] Update ci.yml so pr targets become pr-event targets (#11031)
- [MAINTENANCE] Remove timber from CODEOWNERS (#10996)
- [MAINTENANCE] Update aws ci secret vars. (#11035)
- [MAINTENANCE] Revert Update aws ci secret vars. (#11036)
- [MAINTENANCE] Add ci aws creds. (#11037)
- [MAINTENANCE] Point tests to aws buckets in oss account. (#11034)
- [MAINTENANCE] Fix atomic diagnostic observed value renderers when used with datetime (#11033)
1.3.10
- [DOCS] beta badge followup (#11015)
- [DOCS] Test coverage metrics (#11002)
- [MAINTENANCE] refactor to use data_context fixture (#10997)
- [MAINTENANCE] Remove unneeded tests (#11007)
- [MAINTENANCE] remove numeric dqi from validity expectations (#11005)
- [MAINTENANCE] Upgrade transitive dependencies and stop using internal docusaurus functions (#11009)
- [MAINTENANCE] Stop over alerting on failures (#11021)
- [MAINTENANCE] remove outdated walkthrough modal (#11022)
- [MAINTENANCE] Bump mysql max_connections for tests (#11023)
1.3.9
- [FEATURE] Remove batch_id parameter from Metric classes (#10971)
- [FEATURE]
QueryRowCount
metric (#10964) - [FEATURE] ColumnPairValueInSet metric (#10969)
- [FEATURE] MultiColumnValues metric (#10973)
- [BUGFIX] Validation Definition factory add_or_update doesnt allow updating batch definition (#10960)
- [BUGFIX] Fix 3 batch.compute_metrics bugs and enforce len(metrics) == len(results) (#10979)
- [BUGFIX] Ensure Asset.get_batch_definition returns BatchDefinition with ID (#10986)
- [BUGFIX] Raise error when ExpectColumnValuesToBeBetween is run against an unsupported column (#10995)
- [DOCS] make airflow provider discoverable (#10967)
- [DOCS] Adding title to codeblocks in API (#10970)
- [DOCS] Adds script to update links when new docs version is cut (#10976)
- [DOCS] Add README.md to scripts folder (#10982)
- [DOCS] DOC-1045: Beta badge (#10980)
- [DOCS] DSB-1312: Fix validation definition parameters display (#10981)
- [DOCS] temporarily remove link checker (#10992)
- [DOCS] volume change detection (#10927)
- [DOCS] update action docs to include custom field example (#10987)
- [DOCS] update search key (#10994)
- [DOCS] Adding formatting to codeblocks in API (#10985)
- [MAINTENANCE] Remove
Domain
mixin from Metrics API (#10966) - [MAINTENANCE] Remove old workflow. (#10975)
- [MAINTENANCE] Slack notification message - remove duplicated link text, decluttering (#10890) (thanks @data-han)
- [MAINTENANCE] Upgrade mypy to version 1.15.0 (#10988)
- [MAINTENANCE] Change github action event name we exclude from marker tests (#10991)
- [MAINTENANCE] Remove unused domain key (#10983)
- [MAINTENANCE] Analytics event for validation definition runs (#10984)
- [MAINTENANCE] Upgrade ruff to version 0.9.9 (#10990)
- [MAINTENANCE] Remove duplicated configuration (#10989)
- [MAINTENANCE] refactor to use data_context fixture (#10997)
1.3.8
- [FEATURE]
Batch.compute_metrics()
(#10950) - [FEATURE]
ColumnValuesNonNull
andColumnValuesNonNullCount
metrics (#10959) - [FEATURE] Add mean to metrics API (#10961)
- [FEATURE] Remove batch_id parameter from Metric classes (#10971)
- [DOCS] SSO cta (#10872)
- [DOCS] Display args, returns and raises as tables in API reference pages (#10910)
- [DOCS] typo (#10965)
- [DOCS] Args and Raises in API reference should be able to display several rows (#10968)
- [MAINTENANCE] Remove
table
parameter from all MetricDomain
s (#10954) - [CONTRIB] remove table domain key (#10956) (thanks @VolkovGeoPhy)
1.3.7
- [FEATURE] BatchRowCount Metric (#10944)
- [BUGFIX] remove unused domain key (#10951)
- [DOCS] Replacing search keys with env variables (#10940)
- [DOCS] SSO cta (#10872)
- [MAINTENANCE] Make
Metric.config
un-instantiable and excluded from auto-complete (#10938) - [MAINTENANCE] Drop jinja2 v2 support (#10941)
- [MAINTENANCE] Require
Metric.name
instead of usingname
inference (#10953)
1.3.6
- [BUGFIX] Quote password before passing to SnowflakeURL (#10919)
- [BUGFIX] Trim the unexpected rows query (#10923)
- [BUGFIX] Make
MetricConfiguration.id
immutable (#10929) - [BUGFIX]
test_diagnostic_checklist
import error (#10934) - [BUGFIX] ExpectTableRowCountToBeBetween fails validation with Runtime Parameters (#10925) (thanks @eric-brady)
- [DOCS] Fix HTML entity character in API reference code blocks (#10915)
- [DOCS] AI-recommended Expectations (#10913)
- [DOCS] Fix lack of formatting in actions.py (#10918)
- [DOCS] Feedback form issues should have status "intake" (#10933)
- [DOCS] Fix sidebar highlight by deleting subpages from the sidebar (#10903)
- [DOCS] Added useLocation mock (#10939)
- [MAINTENANCE] Ensure clear error if cloud mode requested and env vars aren't found (#10916)
- [MAINTENANCE] Remove concurrency block from GitHub CI workflow (#10930)
- [MAINTENANCE] Add
Metric
andDomain
base classes (#10920) - [MAINTENANCE] Remove unreferenced docs snippets (#10937)
- [MAINTENANCE] Metric Result types (#10932)
- [MAINTENANCE] Make
Metric.config
un-instantiable and excluded from auto-complete (#10938)
1.3.5
- [DOCS] New icons for admonitions (#10899)
- [DOCS] Algolia fix (#10907)
- [MAINTENANCE] Add
row_condition
datetime testing for Pandas and Spark (#10892) - [MAINTENANCE] Cleanup bigquery every 3 hours (#10900)
- [MAINTENANCE] Add
strict
to Window type (#10906) - [MAINTENANCE] ensure run_id on ValidationDefinition.run (#10909)
1.3.4
- [BUGFIX]
row_condition
datetimes getting tuncated to dates (#10891) - [DOCS] restore lychee (#10889)
- [DOCS] DOC-966: display properties in api reference as tables (#10880)
- [DOCS] DOC-966: display api reference methods signatures as code blocks (#10882)
- [DOCS] Dynamic Parameters for Completeness Expectations (#10873)
- [DOCS] Deleting versioned pages of 0.18 that should not be versioned (#10893)
- [MAINTENANCE] Allow forked PRs to run CI (#10894)
- [MAINTENANCE] Remove bot check on permissions checker action (#10895)
1.3.3
- [BUGFIX] Make validation results' describe_dict return a serializable dict (#10863)
- [BUGFIX] Allow adding Expectations with identical attributes to Suites (#10884)
- [DOCS] Add titles to properties and methods sections in API Reference (#10821)
- [DOCS] Update documentation around batch_parameters and link to updated API docs. (#10877)
- [DOCS] Corrections and clarifications for result_format (#10875)
- [MAINTENANCE] Handle
aws-chunked
encoding type data inTupleS3StoreBackend
(#10861) - [MAINTENANCE] Add user-agent-str to analytics events (#10869)
- [MAINTENANCE] Add method to set analytics user_agent_str (#10883)
- [MAINTENANCE] Convert BatchTestSetup.asset property to BatchTestSetup… (#10864)
- [MAINTENANCE] Migrate from
databricks-sql-connector
todatabricks-sqlalchemy
in tests (#10886)
1.3.2
- [FEATURE] CheckpointFactory.add_or_update (#10856)
- [FEATURE] Enable strict_min/max for ExpectTableRowCountToBeBetween (#10845)
- [BUGFIX]
ExpectColumnUniqueValueCountToBeBetween
strict_min
andstrict_max
not getting set (#10835) - [BUGFIX] Extend existing version logic for pandas
series.between()
inclusive
to missing conditions (#10837) - [BUGFIX] Add input args validators to various Expectations (#10833)
- [DOCS] Expectation selection, group by data quality issue (#10806)
- [DOCS] Editing Admonition titles (#10813)
- [DOCS] consolidate info about analytics / usage stats (#10853)
- [DOCS] update checkpoint snippet location (#10854)
- [DOCS] In-app agent request (#10836)
- [DOCS] small fixes for newer DQ use case articles (#10847)
- [DOCS] Adding keys to duplicated headers (#10846)
- [DOCS] fix custom SQL Expectation approach for cloud (#10844)
- [DOCS] update screenshot of observed value (#10867)
- [MAINTENANCE] Add
--force-reinstall
flag toinvoke deps
and update help text (#10834) - [MAINTENANCE] Update
responses
pin due to mypy error with latest release (#10842) - [MAINTENANCE] Remove xfailed test (#10843)
- [MAINTENANCE] Pin
boto3
due to breaking change (#10862) - [MAINTENANCE] Change mssql tests to use version 18 driver (#10868)
- [MAINTENANCE] Temporary fix to move past failing schema cleanup (#10860)
- [MAINTENANCE] Unpin
snowflake-sqlalchemy
(#10838) - [MAINTENANCE] CI skips xfailed quoted identifier tests (#10857)
- [MAINTENANCE] Enforce a minimum length of 2 for MulticolumnMapExpectation
column_list
(#10850)
1.3.1
- [BUGFIX] Ensure datetime.time can be serialized to JSON (#10795)
- [BUGFIX] Stop overwriting template with description (#10826)
- [DOCS] restore Lychee link checking (#10797)
- [DOCS] replace
context.sources
withcontext.data_sources
(#10794) - [DOCS] ADR around public API docstrings (#10798)
- [DOCS] Change home banner buttons (#10804)
- [DOCS] Remove uppercase transformation for alerts' titles (#10800)
- [DOCS] Adding commonroom web tracking snippet (#10805)
- [DOCS] Adding/deleting other supported databases (#10812)
- [DOCS] Update Learn data pipeline tutorial page (#10828)
- [MAINTENANCE] Implement suite factory add_or_update (#10796)
- [MAINTENANCE] add expectation params to ColumnMapExpectation
mostly
field (#10829) - [MAINTENANCE] Bump
posthog
to V3 (#10814) - [MAINTENANCE] Update expectations to use canonical data quality issues (#10807)
- [MAINTENANCE] Nightly cleanup of stray bigquery schemas (#10815)
- [MAINTENANCE] Enforce mandatory docstrings for public API decorated objects (#10799)
- [MAINTENANCE] Add bigquery requirements file to bigquery cleanup script (#10819)
- [MAINTENANCE] Fix cleanup script (#10820)
- [MAINTENANCE] Enforce comments with linter ignores (#10817)
- [MAINTENANCE] Add to git blame ignore rev (#10822)
- [MAINTENANCE] Ignore Marshmallow V4 warnings (#10825)
- [MAINTENANCE] Change logger warning to debug (#10790)
- [MAINTENANCE] Better logging around and only calling cleanup when we have old BigQuery schemas (#10824)
- [MAINTENANCE] Add
context.validation_definitions.add_or_update
support (#10818)
1.3.0
- [BUGFIX] Handle expectation description from cloud (#10768)
- [BUGFIX] Renderer for UnexpectedRowsExpectation (#10758)
- [BUGFIX] Fix format of UnexpectedRowsExpectation observed_value to be int (#10777)
- [BUGFIX] Metric table.column_type should properly evaluate for snowflake (#10776)
- [BUGFIX] Fix expectation description rendering in DataDocs (#10789)
- [BUGFIX]
Databricks
Fix Type Translation -ExpectColumnValuesToBeInTypeList
andExpectColumnValuesToBeInType
(#10791) - [BUGFIX] Metric
table.column_type
should properly evaluate for Postgres (#10793) - [DOCS] Clarify Connect GX Cloud landing page (#10761)
- [DOCS] Remove
print(validation_results.result_url)
as it isn't supported (#10760) - [DOCS] SUJ-E (#10665)
- [DOCS] Fix underline in links on inline code (#10783)
- [DOCS] Remove unnecessary escape character in Expectation for Gallery (#10780)
- [DOCS] Custom Actions (#10772)
- [DOCS] Schema change detection (#10755)
- [DOCS] Update public_api to include
ValidationAction
components (#10752) - [MAINTENANCE] Change ci pipeline to pull_request_target with a permissions check to allow CI on forks (#10467)
- [MAINTENANCE] Revert "[MAINTENANCE] Change ci pipeline to pull_request_target with a permissions check to allow CI on forks" (#10773)
- [MAINTENANCE] Minor code clean up and refactor around column type exp… (#10764)
- [MAINTENANCE] Diagram on how multi-datasource test setup works (#10766)
- [MAINTENANCE] Add
UnexpectedRowsExpectation
observed value renderer (#10779) - [MAINTENANCE] Remove
docs_link_checker.py
(#10781) - [MAINTENANCE] Add ENUM for Data Quality Issues, Update Core Expectations Docstrings/Schemas (#10759)
- [MAINTENANCE] Adding more test cases for snowflake types (#10786)
- [MAINTENANCE] Update
tasks.py
to remove reference toisort
(#10782) - [MAINTENANCE] Remove GX Cloud onboarding script (#10785)
- [MAINTENANCE] Check filepath existence when evaluating public API report (#10754)
- [MAINTENANCE] Add EventBridge Scheduler service to
cloud-tests
(#10774) - [MAINTENANCE] Deprecate
DataContext.add_or_update_datasource
(#10784) - [MAINTENANCE] Allow
CheckpointResult
andActionContext
to be importable from top-level checkpoint module (#10788) - [MAINTENANCE] Clean up
cloud-tests
environment variables (#10792) - [MAINTENANCE] Adding
databricks
compatibility types (#10787)
1.2.6
- [BUGFIX] Enable custom actions in V1 (#10743)
- [BUGFIX] Fix like pattern expectations to work without optional SQL deps (#10745)
- [BUGFIX] Default
exact_match
toTrue
forExpectTableColumnsToMatchSet
(#10746) - [BUGFIX] check for variance in validator and batch asset (#10744)
- [DOCS] Update Row Condition guidance (#10736)
- [DOCS] Update banner content (#10747)
- [DOCS] key-pair auth for Snowflake (#10751)
- [MAINTENANCE] Add
JOIN
test cases forUnexpectedRowsExpectation
(#10733) - [MAINTENANCE] Enable custom table names in expectation testing framework (#10724)
- [MAINTENANCE] Prefix schema (#10742)
- [MAINTENANCE] Atomic diagnostic observed value renderer for
Expect table columns to match set
(#10748) - [MAINTENANCE] Simplify bigquery assertion (#10750)
1.2.5
- [FEATURE]
expect_table_columns_to_match_ordered_list
observed value renderer (#10683) - [FEATURE] Value-set Expectation observed value renderers (#10718)
- [BUGFIX]
UnexpectedRowsExpectation
- Unable to use{batch}
keyword with partitioner for some backends (#10721) - [DOCS] Cloud Expectation condition docs (#10690)
- [DOCS] DSB-1233: addyng non-versioned-section styling to all the cloud section (#10694)
- [DOCS] Ensure "Connect GX Cloud to ..." pages accurately reflect the current workflow (#10689)
- [DOCS] DSB-1219: Adding redirects (#10692)
- [DOCS] Add databricks to supported datasources in corresponding expectations (#10691)
- [DOCS] Update Manage Data Assets page to reflect the current UI and remove duplicate content (#10695)
- [DOCS] Changing announcement banner (#10703)
- [DOCS] data quality uniqueness article (#10584)
- [DOCS] remove abandoned pages (#10704)
- [DOCS] add code block titles to avoid overlaps in display (#10708)
- [DOCS] Fixing close button on Announcement bar (#10717)
- [DOCS] link fixes - December 2024 (#10716)
- [DOCS] Make announcement bar not closable (#10719)
- [DOCS] - Wrong dependency included in Set Up a GX Environment (#10722)
- [DOCS] data quality -- freshness (#10612)
- [DOCS] fix batch definition data types (#10723)
- [DOCS] fix docs typo (#10725)
- [MAINTENANCE] Pin
snowflake-sqlalchemy
due to breaking change (#10698) - [MAINTENANCE] Add atomic diagnostic renderer to
ExpectColumnMostCommonValueToBeInSet
(#10697) - [MAINTENANCE] Add tests around expectations (#10688)
- [MAINTENANCE] Write tests for misconfigured expectations error reporting (#10696)
- [MAINTENANCE] Test expectations against Snowflake column types (#10706)
- [MAINTENANCE] Test remaining expectations (#10715)
- [MAINTENANCE] Logging message in Datasource store (#10729)
- [MAINTENANCE] Test postgres type support (#10727)
- [CONTRIB] Handle connection error during version check (#10720) (thanks @stejin)
1.2.4
- [DOCS] ADR around not using meta fields (#10672)
- [DOCS] Update Expectation conditions docs (#10661)
- [DOCS] community contributions - November 2024 (#10681)
- [DOCS] Data quality on integrity (#10583)
- [MAINTENANCE] : standardize _atomic_diagnostic_observed_value (#10643)
- [MAINTENANCE] Add support to framework for spark integration testing (#10670)
- [MAINTENANCE] Reduce test duration of flakey test (#10663)
- [MAINTENANCE] DatabricksSQL expectation testing (#10653)
- [MAINTENANCE] Add datetime inference to test framework (#10666)
- [MAINTENANCE] Add bigquery to marker tests (#10674)
- [MAINTENANCE] Update testing framework to work with bigquery (#10675)
- [MAINTENANCE] Fixture to get assets for test framework (#10673)
- [MAINTENANCE] Update CODEOWNERS with reqs owner. (#10684)
- [MAINTENANCE] Remove commented out code (#10686)
- [MAINTENANCE] Tests to prove sqlite partitioners work (#10676)
- [MAINTENANCE] Condition parser typing and row conditions testing (#10667)
- [MAINTENANCE] Improve experience around expectation deletion with Cloud-backed suites (#10662)
1.2.3
- [BUGFIX] double-sided z score renderer (#10648)
- [BUGFIX] Stop masking config strings (#10625)
- [DOCS] Mark actions as not ready for contributions (#10646)
- [DOCS] MS Teams Action (#10655)
- [DOCS] Remove install instructions for redshift and trino (#10660)
- [MAINTENANCE] Clean up Azure CI configuration (#10638)
- [MAINTENANCE] add Aaron to teams file (#10641)
- [MAINTENANCE] Bump ruff to
0.7.2
(#10629) - [MAINTENANCE] Bump
docstring-parser
to0.16
(#10608) - [MAINTENANCE] Add integration test for
MicrosoftTeamsNotificationAction
(#10628) - [MAINTENANCE] Clean up
MicrosoftTeamsNotificationAction
docstring and import patterns (#10642) - [MAINTENANCE] Add constant for all unparameterized data sources to test (#10654)
- [MAINTENANCE] Refactor Checkpoint utils (#10649)
- [MAINTENANCE] Sqlite integration testing (#10657)
- [MAINTENANCE] Use random schema in integration tests (#10658)
1.2.2
- [FEATURE] Add check for valid column type when calling add_batch_def in a sql asset (#10590)
- [FEATURE] Expectations tests against SQL backends infer column types (#10622)
- [BUGFIX] Parse spaces in row condition column name (#10611)
- [BUGFIX] Batch Expectations correctly handle
date
min and max values (#10613) - [BUGFIX] Mask conn_str in configs (#10626)
- [BUGFIX] Missing renderer params when
row_condition
is used (#10632) - [DOCS] Updated cloud action support posture to be in line with reality (#10609)
- [DOCS] Add dynamic parameters documentation (#10483)
- [DOCS] Fix column name in failing example for ExpectColumnValuesToBeB… (#10620)
- [DOCS] remove unsupported actions (#10624)
- [MAINTENANCE] Bump mermaid from 10.9.0 to 10.9.3 in /docs/docusaurus (#10549)
- [MAINTENANCE] Bump http-proxy-middleware from 2.0.6 to 2.0.7 in /docs/docusaurus (#10566)
- [MAINTENANCE] Mock posthog in action tests (#10615)
- [MAINTENANCE] Add another member to teams.yml (#10616)
- [MAINTENANCE] Validate Expectation JSON Schema follows meta-schema specification (#10627)
- [MAINTENANCE] Prevent unneeded test setup/teardown (#10619)
- [MAINTENANCE] dynamically generate extra table names in tests (#10630)
- [MAINTENANCE] test cleanup (#10631)
- [MAINTENANCE] Add mysql support to testing framework (#10633)
- [MAINTENANCE] Make PR Title Checker GH Action more secure (#10636)
- [MAINTENANCE] Add mssql support to testing framework (#10634)
1.2.1
- [BUGFIX] Add redirect for deploy-gx-agent (#10573)
- [BUGFIX] Fix typing on
mostly
andvalue_set
fields (#10571) - [BUGFIX]
ExpectationSuite
equality should ignore expectation ordering (#10562) - [BUGFIX] Get
MicrosoftTeamsNotificationAction
working with V1 (#10593) - [BUGFIX] Support string substitution for EmailAction (#10600)
- [BUGFIX] Ensure that data docs are accessible through MS Teams action results (#10599)
- [BUGFIX] Incorrect row count when
UnexpectedRowsExpectation
returns more than 200 rows (#10604) - [DOCS] Edit API reference internal links to prevent 404s (#10528)
- [DOCS] data quality distribution analysis, working draft (#10440)
- [DOCS] Changing broken links in changelog file and some left in reference/API (#10588)
- [DOCS] Fixes small friction points noticed during onboarding (#10560)
- [DOCS] remove duplication in GCP Secret Manager instructions (#10591)
- [DOCS] loose ends for python 3.12 support (#10596)
- [DOCS] Fixes more small issues noticed during onboarding (#10598)
- [DOCS] Mark
MicrosoftTeamsNotificationAction
as first-class (#10595) - [DOCS] Add lychee to spot broken urls (#10585)
- [MAINTENANCE] PostgreSQL Expectation Tests (#10582)
- [MAINTENANCE] Snowflake Expectation Tests (#10586)
- [MAINTENANCE] Reenable xfailed e2e tests (#10555)
- [MAINTENANCE] Add test to public api to ensure that decorated methods have decorated parent classes (#10529)
- [MAINTENANCE] Bump
ruff
andmypy
versions to0.7.1
and1.13.0
, respectively (#10565) - [MAINTENANCE] Return informative error when saving expectation suite (#10570)
- [MAINTENANCE] Filesystem expectation testing (#10556)
- [MAINTENANCE] Update teams.yml devrel members. (#10567)
- [MAINTENANCE] Update 0.18.x changelog for release of 0.18.22 (#10575)
- [MAINTENANCE] Add more canonical expectation tests (#10578)
- [MAINTENANCE] Clean up misc utils (#10581)
- [MAINTENANCE] update docs link for configuring credentials (#10580)
- [MAINTENANCE] DRY up config substitution for slack notifications (#10602)
- [MAINTENANCE] Support config substitution for
MicrosoftTeamsNotificationAction
(#10606) - [MAINTENANCE] Add testing support for multi-asset data sources (#10592)
- [MAINTENANCE] Instrument analytics for Checkpoint action creation and runs (#10597)
1.2.0
- [BUGFIX] Remove
row_condition
from Expectations for which it does not apply (#10519) - [BUGFIX] fix rendering performance (#10530)
- [BUGFIX] Ensure that file-backed domain objects are stored in JSON files (#10523)
- [BUGFIX] File Path Batch Definitions serialize incorrectly (#10543)
- [DOCS] Bump max Python version in docs (#10522)
- [DOCS] Fix some typos (#10521)
- [DOCS] Add redirects for old URLs that no longer exist (#10516)
- [DOCS] Update Data Doc site configuration page (#10536)
- [DOCS] Add base
Datasource
to Public API (#10527) - [DOCS] DSB-1009: Add Learn page for GX - Airflow data pipeline tutorial (#10534)
- [DOCS] Change href from relative path to root in API docs (version 0.18) (#10507)
- [DOCS] Fixing broken links (#10541)
- [DOCS] Community/issues board mentions (#10548)
- [DOCS] Update _redirects (#10558)
- [MAINTENANCE] ruff 0.6.9 + mypy 0.12 (#10525)
- [MAINTENANCE] Bump
ruff
to0.7.0
(#10535) - [MAINTENANCE] Add
DataAsset.get_batch_definition
to public API docs (#10533) - [MAINTENANCE] Remove
TRY203
Ruff violations (#10540) - [MAINTENANCE] Update gx row_condition parser (#10524)
- [MAINTENANCE] Use python 3.12 in netlify (#10531)
- [MAINTENANCE] Update airflow snippet to use checkpoint name rather than id (#10551)
- [MAINTENANCE] Fix azure aws/spark tests (#10550)
- [MAINTENANCE] Remove util functions from public API (#10557)
- [MAINTENANCE] Basic expectation testing framework (#10554)
- [MAINTENANCE] Reenable xfailed e2e tests (#10555)
1.1.3
- [BUGFIX] Fix datadocs icons (#10511)
- [DOCS] Add
public_api
decorators to object factories (#10513) - [DOCS] Patch incorrect label in migration guide (#10518)
- [MAINTENANCE] Remove
snapshottest
dependency (#10498) - [MAINTENANCE] Add GH Actions for release process (#10502)
- [MAINTENANCE] Support Python 3.12 (#10503)
- [MAINTENANCE] Patch
ge_releaser
installation pattern in released-related GH Actions (#10509) - [MAINTENANCE] Enable pact test for all expectation suites (#10506)
- [MAINTENANCE] Isolate
clickhouse
marker in CI (#10512)
1.1.2
- [FEATURE] Add context.enable_analytics method (#10385)
- [BUGFIX] V1 Validator shouldn't mutate user's result format dict (#10496)
- [DOCS] Update min version in
_data.jsx
(#10494) - [MAINTENANCE] Remove
result_url
fromCheckpointResult
(#10493) - [MAINTENANCE] Deprecate context.get_datasource (#10471)
1.1.1
- [BUGFIX] Ensure that
SlackNotificationAction
credentials don't get serialized (#10476) - [BUGFIX] Ensure that Cloud-backed validation results get IDs (#10478)
- [DOCS] Remove Python 3.8 reference from readme (#10474)
- [DOCS] dbt Tutorial created in collaboration with Jonathan Porter (#10458)
- [DOCS] Replace terms.json urls in v0.18 (#10479)
- [DOCS] V0 to V1 Migration Guide (#10477)
- [MAINTENANCE] Remove Python 3.8 Support (EOL) (#10441)
- [MAINTENANCE] Bump dompurify from 3.0.11 to 3.1.7 in /docs/docusaurus (#10465)
- [MAINTENANCE] Bump express from 4.19.2 to 4.21.0 in /docs/docusaurus (#10464)
- [MAINTENANCE] Bump webpack from 5.88.2 to 5.94.0 in /docs/docusaurus (#10463)
- [MAINTENANCE] Bump micromatch from 4.0.5 to 4.0.8 in /docs/docusaurus (#10466)
- [MAINTENANCE] Remove old team and non-gx employees from teams.yml (#10469)
- [MAINTENANCE] Remove contrib pipeline. (#10470)
- [MAINTENANCE] Remove
makefun
dependency (#10472) - [MAINTENANCE] Add
public-api
check back to CI (#10449) - [MAINTENANCE] Clean up requirements files (#10485)
- [MAINTENANCE] Remove
ipython
dependency (#10487) - [MAINTENANCE] Remove
pytz
dependency (#10489) - [MAINTENANCE] Update experimental metric repository for V1 backend api (#10486)
- [MAINTENANCE] Remove
urllib3
dependency (#10488)