Stefan Mikic's Tech RadarStefan Mikic's Tech Radar

✅ Great Expectations

data-qualitytesting
Hold

Short description:

Great Expectations is an open-source Python-based data validation framework that allows teams to assert expectations on data quality and pipeline reliability through tests and documentation.

Why it's on hold:

Great Expectations has been placed on hold due to a combination of practical and strategic constraints. While the tool offers a robust and expressive framework for asserting data quality, several friction points have emerged during evaluation:

  • Complex Integration Overhead: Integrating Great Expectations into custom pipelines proved cumbersome.

  • Maintenance Burden: The YAML-based configuration model, though flexible, becomes verbose and difficult to manage at scale. Small changes in schema or expectations often require disproportionate updates across multiple config files, leading to operational brittleness.