Skip to content

CHANGELOG.md

Latest commit

 

History

History
1766 lines (954 loc) · 62.7 KB

CHANGELOG.md

File metadata and controls

1766 lines (954 loc) · 62.7 KB
Raw

Changelog

All notable changes to dataprof are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

For the user-oriented overview, upgrade checklist, and migration details, read the 0.12.0 release notes.

[0.12.0] - 2026-09-30

Added

  • Report numbers written with a decimal comma or digit grouping (#807) by @AndreaBozzo

  • Decide sampled quality scores on a 99.9% interval over the scan (#789) by @AndreaBozzo

  • Add structured findings to ProfileReport (#770) by @AndreaBozzo

  • Record metric semantics in saved reports (#765) by @AndreaBozzo

  • Benchmarks: Measure energy and peak memory (#748) by @AndreaBozzo

  • Benchmarks: Unify suites and publish reproducible evidence (#736) by @AndreaBozzo

  • Record engine and CSV recovery provenance (#735) by @AndreaBozzo

  • Python: Remove flat quality accessors (#733) by @AndreaBozzo

  • Python: Add quality gate module entrypoint (#732) by @AndreaBozzo

  • Add a batch quality-gate API with structured results (#725) by @AndreaBozzo

  • Python: Accept Arrow C Stream inputs (#706) by @AndreaBozzo

  • Partial: Report whole-file counters in a Parquet structural report (#702) by @AndreaBozzo

  • Add column projection across profiling paths (#655) by @QinXi-ai

Changed

  • Run benchmarks from an empty Criterion directory by @AndreaBozzo

  • Stop PR runs from evicting master's caches and building the extension twice (#795) by @AndreaBozzo

  • Throttle per-chunk peak memory sampling (#775) by @AndreaBozzo

  • Raise the Examples Smoke timeout to 30 minutes (#773) by @AndreaBozzo

  • Columnar: Read the ragged-row count from arrow-csv (#754) by @AndreaBozzo

  • Measure Python and Arrow boundary costs (#750) by @AndreaBozzo

  • Distinguish benchmark startup, first use, and steady state (#747) by @AndreaBozzo

  • Python: Unify native and restored report accessors (#746) by @AndreaBozzo

  • Fuzz: Add bounded parser and report fuzz targets (#734) by @AndreaBozzo

  • Update rustls by @AndreaBozzo

  • Python: Split bindings into private modules (#705) by @AndreaBozzo

  • Tests: Split Python API test suite (#704) by @AndreaBozzo

  • Compile the advertised MSRV feature graphs on 1.96 (#684) by @AndreaBozzo

  • Run omitted database regression suites (#683) by @AndreaBozzo

  • Arrow: Remove unreachable CSV dispatch (#679) by @AndreaBozzo

  • Metrics: Compute numeric aggregates with a stable accumulator (#677) by @AndreaBozzo

Dependencies

  • Deps: Bump jsonschema from 0.56.0 to 0.57.0 (#796) by @dependabot[bot]

  • Deps: Bump benchmark setuptools to 81.0.0 by @dependabot[bot]

  • Deps: Bump benchmark polars to 1.44.2 by @dependabot[bot]

  • Deps: Bump ty to 0.0.81 by @dependabot[bot]

  • Deps: Bump polars to 1.44.2 by @dependabot[bot]

  • Deps: Bump ruff to 0.16.7 by @dependabot[bot]

  • Deps: Bump uuid to 1.26.1 by @dependabot[bot]

  • Deps: Bump wide to 1.7.1 by @dependabot[bot]

  • Deps: Bump toml to 1.1.6 by @dependabot[bot]

  • Deps: Bump arrow and parquet to 60.0.0 (#751) by @AndreaBozzo

  • Deps: Bump github/codeql-action from 4.37.9 to 4.38.0 (#745) by @dependabot[bot]

  • Deps: Bump jsonschema from 0.55.1 to 0.56.0 (#726) by @dependabot[bot]

  • Deps: Bump toml from 1.1.4+spec-1.1.0 to 1.1.5+spec-1.1.0 (#687) by @dependabot[bot]

  • Deps: Bump jsonschema from 0.52.1 to 0.55.0 (#688) by @dependabot[bot]

  • Deps-dev: Bump polars from 1.43.2 to 1.44.1 (#689) by @dependabot[bot]

  • Deps-dev: Bump ty from 0.0.75 to 0.0.78 (#690) by @dependabot[bot]

  • Deps-dev: Bump ruff from 0.16.5 to 0.16.6 (#691) by @dependabot[bot]

  • Deps: Bump github/codeql-action from 4.37.8 to 4.37.9 (#659) by @dependabot[bot]

  • Deps-dev: Bump ty from 0.0.71 to 0.0.75 (#634) by @dependabot[bot]

  • Deps-dev: Bump maturin from 1.14.1 to 1.15.0 (#635) by @dependabot[bot]

  • Deps-dev: Bump ruff from 0.16.3 to 0.16.5 (#636) by @dependabot[bot]

  • Deps: Bump log from 0.4.33 to 0.4.34 (#633) by @dependabot[bot]

  • Deps: Bump wide from 1.6.1 to 1.7.0 (#632) by @dependabot[bot]

  • Deps: Bump jsonschema from 0.49.9 to 0.52.0 (#631) by @dependabot[bot]

  • Deps: Bump uuid from 1.24.0 to 1.26.0 (#630) by @dependabot[bot]

  • Deps: Bump github/codeql-action from 4.37.3 to 4.37.8 (#616) by @dependabot[bot]

Documentation

  • Terse README, 0.12 landing page, and a readable benchmark page (#831) by @AndreaBozzo

  • Python: Document column projection (#695) by @AndreaBozzo

  • Ci: Explain contributor decode audit (#681) by @AndreaBozzo

Fixed

  • Profile decimal columns at their value, not their unscaled integer (#828) by @AndreaBozzo

  • Bound uniqueness with certainty past a million distinct rows (#820) by @AndreaBozzo

  • Validate US phone numbers against the numbering plan (#818) by @AndreaBozzo

  • Read a column's slash dates in one day/month order (#817) by @AndreaBozzo

  • Run Welford on shifted values so a large offset keeps the spread exact (#806) by @AndreaBozzo

  • Report the mean of values whose sum overflows f64, not NaN (#804) by @AndreaBozzo

  • Report an IQR that overflows f64 as null, not inf (#803) by @AndreaBozzo

  • Report a variance that overflows f64 as null, not 0.0 (#800) by @AndreaBozzo

  • Compare start and end dates only when they come from the same row (#792) by @AndreaBozzo

  • Report CSV sources that end inside a quoted field (#791) by @AndreaBozzo

  • Count distinct values exactly up to a million on every engine (#786) by @AndreaBozzo

  • Cap approximate distinct counts at the values seen (#779) by @AndreaBozzo

  • Report progress on every file route, including the default engine (#777) by @AndreaBozzo

  • Python: Make the async database helpers coroutine functions (#778) by @AndreaBozzo

  • Stop byte-capped CSV scans at the first record past the budget (#776) by @AndreaBozzo

  • Seed the numeric reservoir so in-memory profiles are deterministic (#774) by @AndreaBozzo

  • Python: Refuse text and overflowing numbers as policy thresholds (#772) by @AndreaBozzo

  • Profile nested columns by their counts, not a display string (#769) by @AndreaBozzo

  • Python: Reject malformed score weights in flat summaries (#764) by @AndreaBozzo

  • Python: Keep custom score weights in the flat summary (#761) by @AndreaBozzo

  • Persist one canonical report document (#759) by @AndreaBozzo

  • Metrics: Align text frequency absence across input paths (#758) by @AndreaBozzo

  • Columnar: Stop the CSV decode at the row cap (#755) by @AndreaBozzo

  • Benchmarks: Preserve evidence on failure (#740) by @AndreaBozzo

  • Align empty-source quality across input paths (#724) by @AndreaBozzo

  • Runtime: Record why a report carries no quality assessment (#722) by @AndreaBozzo

  • Parquet: Spread capped samples across files (#721) by @AndreaBozzo

  • Metrics: Scope and enforce the cross-engine numeric equality contract (#708) by @AndreaBozzo

  • Ci: Enforce published Python interpreters (#707) by @AndreaBozzo

  • Agent: Redact host paths in model context (#703) by @AndreaBozzo

  • Partial: Refuse duplicate Parquet column names, and name the type's source (#701) by @AndreaBozzo

  • Partial: Type Parquet text columns from their values, not the metadata (#699) by @AndreaBozzo

  • Streaming: Preserve EOF metadata for async stops (#696) by @atirna

  • Omit statistics when no values parsed (#682) by @AndreaBozzo

  • Python: Document supported engine aliases (#680) by @AndreaBozzo

  • Harden numeric statistics and Python input boundaries (#672) by @AndreaBozzo

  • Metrics: Profile timestamp columns in named time zones (#669) by @AndreaBozzo

  • Python: Profile a zero-row Arrow source over its declared columns (#666) by @AndreaBozzo

  • Python: Profile every chunk of a pandas or polars DataFrame (#662) by @AndreaBozzo

  • Parquet: Profile a column by its values, not its physical encoding (#661) by @AndreaBozzo

  • Metrics: Accept RFC 3339 offsets, on one date grammar for every engine (#660) by @AndreaBozzo

  • Metrics: Count identical JSON records as duplicates across a late key (#658) by @AndreaBozzo

  • Runtime: Make accumulator merge produce the single-pass profile (#650) by @AndreaBozzo

  • Database: Decode MySQL TIME columns as a time of day, not a duration (#644) (#646) by @AnandkumarMall

  • Database: Decode temporal, decimal, UUID and unsigned columns (#642) by @AndreaBozzo

  • Metrics: Count text lengths in Unicode scalar values, not UTF-8 bytes (#641) by @AndreaBozzo

  • Metrics: Withhold evidence for dimensions that assessed nothing (#640) by @AndreaBozzo

  • Metrics: Report no overall quality score when nothing was assessed (#629) by @AndreaBozzo

  • Preserve numeric and CSV profiling correctness (#628) by @AndreaBozzo

[0.11.0] - 2026-08-25

Added

  • Errors: Retain originating errors as a walkable source chain (#617) by @AndreaBozzo

  • Parquet: Profile materialized Parquet bytes through the async path (#582) by @aryansk

  • Schema: Add bytes source_type for byte-buffer inputs (#583) by @aryansk

  • Python: Ship async and HTTP profiling in the default wheel (#595) by @AndreaBozzo

  • Profile: Flag a column that has no dominant type (#578) by @AndreaBozzo

  • Python: Pin the public surface to what all declares (#529) by @AndreaBozzo

  • Agents: Make the dataprof skill accurate, self-checking, and able to grow (#510) by @AndreaBozzo

  • Report: Publish versioned JSON schema (#493) by @AndreaBozzo

Changed

  • Normalize line endings with .gitattributes (#618) by @AndreaBozzo

  • Pin the Rust toolchain and fix the clippy lint 1.98 added (#615) by @AndreaBozzo

  • Release: Single-source the shipped wheel feature list (#594) by @aryansk

  • Give feature-checks room to survive a cold cache (#597) by @AndreaBozzo

  • Stop building the Rust extension in the Python lint job (#596) by @AndreaBozzo

  • Stop the wheel smoke build compiling the tree twice (#533) by @AndreaBozzo

  • Label pull requests by changed path (#508) by @AndreaBozzo

Dependencies

  • Deps-dev: Bump ruff from 0.16.1 to 0.16.3 (#603) by @dependabot[bot]

  • Deps-dev: Bump pyarrow from 25.0.0 to 25.0.1 (#602) by @dependabot[bot]

  • Deps: Bump thiserror from 2.0.19 to 2.0.20 (#601) by @dependabot[bot]

  • Deps: Bump async-trait from 0.1.91 to 0.1.92 (#600) by @dependabot[bot]

  • Deps: Bump futures from 0.3.33 to 0.3.34 (#599) by @dependabot[bot]

  • Deps: Bump wide from 1.6.0 to 1.6.1 (#598) by @dependabot[bot]

  • Deps: Bump pyo3 from 0.29.0 to 0.29.2 (#567) by @dependabot[bot]

  • Deps-dev: Bump ruff from 0.16.0 to 0.16.1 (#524) by @dependabot[bot]

  • Deps: Bump parquet from 59.1.0 to 59.2.0 (#566) by @dependabot[bot]

  • Deps-dev: Bump ty from 0.0.65 to 0.0.71 (#570) by @dependabot[bot]

  • Deps-dev: Bump polars from 1.39.0 to 1.43.2 (#569) by @dependabot[bot]

  • Deps: Bump jsonschema from 0.49.2 to 0.49.6 (#568) by @dependabot[bot]

  • Deps: Bump arrow from 59.1.0 to 59.2.0 (#565) by @dependabot[bot]

  • Deps: Bump toml from 1.1.3+spec-1.1.0 to 1.1.4+spec-1.1.0 (#522) by @dependabot[bot]

  • Deps: Bump wide from 1.5.0 to 1.6.0 (#523) by @dependabot[bot]

  • Deps: Bump dtolnay/rust-toolchain from 1.96 to 1.100 (#521) by @dependabot[bot]

  • Deps-dev: Bump ty from 0.0.56 to 0.0.65 (#490) by @dependabot[bot]

  • Deps: Bump ipython from 8.38.0 to 8.39.0 (#492) by @dependabot[bot]

  • Deps-dev: Bump ruff from 0.15.6 to 0.16.0 (#491) by @dependabot[bot]

  • Deps: Bump serial_test from 3.5.0 to 4.0.1 (#489) by @dependabot[bot]

Documentation

  • Make the README portable and add a website contributor callout (#625) by @AndreaBozzo

  • Add software citation metadata and rewrite the PR template (#624) by @AndreaBozzo

  • Add compiling Rust examples to the JSON and Parquet entry points (#620) by @AndreaBozzo

  • Record the database decision and describe what the wheel ships (#606) by @AndreaBozzo

  • Add a code-review agent skill for Copilot reviews (#563) by @AndreaBozzo

  • Add recipes for validating saved ProfileReport JSON (#535) by @aryansk

  • Update reference format for AGENTS.md in CLAUDE.md by @AndreaBozzo

Fixed

  • Metrics: Compare real dates in timeliness, not years or raw strings (#621) by @AndreaBozzo

  • Python: Document the source-build recipe and guard the shipped feature set (#619) by @AndreaBozzo

  • Python: Validate Arrow data imported over the C Data Interface (#610) by @AndreaBozzo

  • Python: Profile every batch of a chunked Table, and accept any Arrow PyCapsule producer (#608) by @AndreaBozzo

  • Metrics: Count complete records on rows, not null cells (#607) by @AndreaBozzo

  • Schema: Serialize report mapping fields in deterministic order (#584) by @aryansk

  • Core: Make StreamingError suggestions per-site instead of a hardcoded chunk-size remedy (#581) by @aryansk

  • Python: Assess quality by default on database queries (#604) by @AndreaBozzo

  • Async: Reject duplicate CSV headers (#575) by @aryansk

  • Metrics: Align empty CSV quality parity (#576) by @aryansk

  • Partial: Report schema_stable from scan exhaustion, not the raw cap (#580) by @aryansk

  • Csv: Ignore blank records when scoring delimiter candidates (#579) by @aryansk

  • Patterns: Make locale a closed set instead of a silent filter (#577) by @AndreaBozzo

  • Csv: Align header-only analysis across engines (#572) by @floze-the-genius

  • Partial: Preserve header-only CSV schema (#557) by @floze-the-genius

  • Python: Keep llm null flags stable across round-trips (#556) by @aryansk

  • Metrics: Validate date columns against the forms that typed them (#564) by @AndreaBozzo

  • Metrics: Score string columns against their dominant lexical class (#560) by @AndreaBozzo

  • Agents: Disclose sample scope, and teach the skill that a perfect score is not evidence (#555) by @AndreaBozzo

  • Engines: Count ragged rows on the columnar CSV engine (#543) by @AndreaBozzo

  • Engines: Honour and disclose a row cap of zero (#542) by @AndreaBozzo

  • Json: Define JSONL as one record per physical line on every transport (#540) by @AndreaBozzo

  • Database: Preserve query column order in profile reports (#539) by @AndreaBozzo

  • Database: Apply metric packs and locale on database queries (#538) by @AndreaBozzo

  • Core: Apply metric packs and locale on JSON, Parquet, and async paths (#537) by @AndreaBozzo

  • Json: Report a record with no fields as a row on every transport (#534) by @AndreaBozzo

  • Python: Read Parquet byte buffers natively, without pandas (#532) by @AndreaBozzo

  • Json: Never discard a non-object JSON record (#531) by @AndreaBozzo

  • One rounding convention across the Rust and Python layers (#528) by @AndreaBozzo

  • Python: Restore hint bindings and ratio precision on report round-trip (#527) by @AndreaBozzo

  • Json: Preserve source field order for JSON and JSONL columns (#507) by @AndreaBozzo

  • Accept leading UTF-8 BOM in JSON inputs (#506) by @AndreaBozzo

  • Harden profiling edge-case behavior (#498) by @AndreaBozzo

[0.10.0] - 2026-07-24

Added

  • Json: Unify malformed-record policy across file/bytes/async JSONL (#382) (#457) by @AndreaBozzo

  • Input: Enforce duplicate column-name policy across engines (#381) (#456) by @AndreaBozzo

  • Python: Finish #435 report API ergonomics — profiles, save html/md, low_sample_warning, zero-row semantics (#455) by @AndreaBozzo

  • Errors: Standardize actionable diagnostics and preserve source context (#446) by @AndreaBozzo

  • Hints: Validate semantic hints and record binding evidence (#420) (#441) by @AndreaBozzo

  • Record report execution engine (#438) by @floze-the-genius

  • Metrics: Per-column invalid_count for values excluded from numeric stats (#437) by @AndreaBozzo

  • Report: Version the serialized ProfileReport schema (#415) by @AndreaBozzo

  • Python: Add agent security guard for MCP surfaces (#410) by @AndreaBozzo

  • Capabilities: Add capabilities function to report installed features and metadata (#409) by @AndreaBozzo

  • Contributor: Add smoke test script and update CONTRIBUTING.md (#408) by @AndreaBozzo

  • Metrics: Expose provenance for approximate distinct counts (#406) by @AndreaBozzo

  • Metrics: Add Validity and Precision dimensions (#399) by @AndreaBozzo

  • Metrics: Make quality score weights configurable (#398) by @AndreaBozzo

  • Metrics: Require explicit temporal columns (#397) by @AndreaBozzo

  • Metrics: Honor identifier column hints (#396) by @AndreaBozzo

  • Metrics: Track duplicate rows across streams (#395) by @AndreaBozzo

  • Metrics: Score only assessed dimensions, using every sub-metric (#394) by @AndreaBozzo

  • Site: Rehaul GitHub Pages into a project homepage by @AndreaBozzo

Changed

  • Ci: Parallelize database integration builds (#468) by @AndreaBozzo

  • Metrics: Add table-driven completeness scenarios (#407) by @AndreaBozzo

  • Add issue templates in the directory GitHub actually reads (#390) by @AndreaBozzo

  • Json: Format decode audit by @AndreaBozzo

  • Add cross-engine parity suite (#369) by @AndreaBozzo

  • Make the ty type check blocking now that the codebase passes it by @AndreaBozzo

  • Deny: Document why RUSTSEC-2024-0436 is ignored by @AndreaBozzo

  • Make the benchmark page generator pass ruff format and lint by @AndreaBozzo

Dependencies

  • Deps-dev: Bump pytest from 9.0.3 to 9.1.1 (#478) by @dependabot[bot]

  • Deps-dev: Bump maturin from 1.12.6 to 1.14.1 (#477) by @dependabot[bot]

  • Deps: Bump github/codeql-action from 4.36.1 to 4.37.3 (#476) by @dependabot[bot]

  • Deps: Bump actions/checkout from 6.0.3 to 7.0.1 (#475) by @dependabot[bot]

  • Deps-dev: Bump pyarrow from 23.0.1 to 25.0.0 (#479) by @dependabot[bot]

  • Deps: Bump actions/setup-python from 6 to 7 (#474) by @dependabot[bot]

  • Deps: Bump serde_json from 1.0.149 to 1.0.151 (#450) by @dependabot[bot]

  • Deps: Bump uuid from 1.23.4 to 1.24.0 (#451) by @dependabot[bot]

  • Deps: Bump futures from 0.3.32 to 0.3.33 (#448) by @dependabot[bot]

  • Deps: Bump toml from 0.9.12+spec-1.1.0 to 1.1.3+spec-1.1.0 (#285) by @dependabot[bot]

  • Deps: Bump wide from 1.2.0 to 1.5.0 (#391) by @dependabot[bot]

  • Deps: Bump sysinfo from 0.39.5 to 0.39.6 (#392) by @dependabot[bot]

  • Deps: Bump regex from 1.12.3 to 1.13.0 (#393) by @dependabot[bot]

Documentation

  • Document workspace crate boundaries (#487) by @AndreaBozzo

  • Add "Why dataprof?" comparison guide (#414) by @AndreaBozzo

  • Templates: Drop CLI references and reinforce profile-only boundary by @AndreaBozzo

  • Add AGENTS.md with a CLAUDE.md redirect by @AndreaBozzo

  • Align SECURITY.md supported versions with the 0.9.x line by @AndreaBozzo

Fixed

  • Python: Preserve async byte source sizes (#485) by @AndreaBozzo

  • Json: Validate streaming array grammar (#484) by @AndreaBozzo

  • Json: Reject truncated final JSONL records (#481) by @AndreaBozzo

  • Security: Audit every published feature graph (#464) (#473) by @AndreaBozzo

  • Sampling: Make every strategy sample, on every path (#459) (#472) by @AndreaBozzo

  • Execution: Honor resource controls and correct provenance (#460) (#471) by @AndreaBozzo

  • Async: Count ragged CSV rows and honor CSV options (#462) (#469) by @AndreaBozzo

  • Patterns: Require locale evidence for summary claims (#467) by @AndreaBozzo

  • Harden profiling contracts before 0.10 (#466) by @AndreaBozzo

  • Csv: One clean diagnostic for non-UTF-8 input (#426) (#458) by @AndreaBozzo

  • Partial: Apply multi-offset row-count sampling to JSONL (#454) by @AndreaBozzo

  • Partial: Remove prefix bias from CSV quick_row_count sampling (#453) by @AndreaBozzo

  • Csv: Surface ragged CSV rows instead of silently normalizing them (#452) by @AndreaBozzo

  • Python: Preserve DataFrame column order (#444) by @floze-the-genius

  • Assess inferred dates in timeliness (#445) by @AndreaBozzo

  • Hints: Make streamed binding evidence exact (#443) by @AndreaBozzo

  • Metrics: Base numeric stats from exact accumulators; disclose sampled order statistics (#432) by @AndreaBozzo

  • Csv: Quote-aware delimiter sniffer; absent delimiters can no longer win (#431) by @AndreaBozzo

  • Metrics: Populate execution.memory_peak_mb in incremental and columnar engines (#422) by @AndreaBozzo

  • Metrics: Track duplicate rows in columnar and record-batch engines (#421) by @AndreaBozzo

  • Python: Describe() 50% falls back to median when quartiles are absent (#416) by @AndreaBozzo

  • Metrics: Treat quality_dimensions=[] as quality not analyzed (#411) by @AndreaBozzo

  • Replace columnar 1,000-value unique-count cap with bounded estimator (#389) by @AndreaBozzo

  • Decode: Enforce audited invariants by @AndreaBozzo

  • Decode: Audit swallowed errors on decode paths by @AndreaBozzo

  • Python: Make ty check pass across examples, tests, and stubs by @AndreaBozzo

[0.9.0] - 2026-07-09

See release-notes.md for the narrative 0.9.0 notes and the migration guide.

Added

  • Python: to_llm_context(max_tokens=1000, include_samples=False) — a token-bounded, agent-oriented report summary covering shape, provenance caveats, derived quality flags, schema, and pattern names. Tokens are estimated as ceil(len / 4), so the output is deterministic and needs no tokenizer. Sections are budgeted by priority. Raw cell values are withheld unless include_samples=True. (#331)
  • Python: analyze_structure(path, max_rows=None) — lightweight structural profiling returning a StructureReport, joining infer_schema() and quick_row_count() on the partial analysis surface. (#319)
  • Python: ProfileReport.load(path) — reload a report saved with save() from a .json file path (the path-based counterpart to from_json()/from_dict()). .csv/.parquet paths raise a clear ValueError since they store column profiles only. (#325)
  • Python: report ergonomics — to_html(), to_markdown(), compare(), from_dict(), and from_json(). The compare() return shape is provisional and will be aligned with the Rust QualityDelta type (#310). (#316)
  • Python: dp.profile() accepts dict, list-of-dicts, and bytes inputs on the base wheel, going straight to the Rust core via a new profile_columns entry point rather than adapting over pd.DataFrame. Column order now follows the input, and an integer column containing a null stays an integer instead of being widened to float. (#360)
  • Python: type stubs for the compiled extension (dataprof/_dataprof.pyi), so type checkers and IDEs can resolve the native core. The public stubs now re-export these types instead of duplicating them.

Changed

  • Python: the supported runtime floor is Python 3.10+ (3.8 and 3.9 are past end-of-life). Rust MSRV is now 1.96. Project metadata, tooling, and CI exercise both minimums explicitly. (#326)
  • Python: flat DataQualityMetrics accessors such as missing_values_ratio, key_uniqueness, and outlier_ratio now emit DeprecationWarning. Use nested dimension properties such as quality.completeness, quality.uniqueness, and quality.accuracy, which are None when a dimension was not computed. The flat accessors keep their 0.8-compatible values for the 0.9 line. low_sample_warning and overall_quality_score() are report-level signals and are not deprecated. (#326)
  • Remove panic = "abort" from the release profile. A panic in the Rust core aborted the process and took the Python interpreter with it; panics now unwind to the PyO3 boundary and surface as ordinary Python exceptions.

Fixed

  • Parquet/Arrow: exclude nulls from numeric column statistics. The ten numeric array handlers guarded on if let Some(value) = array.value(index).into(), but impl<T> From<T> for Option<T> makes that arm always taken, so the physical value under a null (0.0) was folded into min, mean, std_dev, and unique_count. This affected every nullable numeric column read through Parquet, pandas, polars, or Arrow. Statistics for such columns will change on upgrade; regenerate any stored baselines. (#358)
  • Python: to_llm_context(include_samples=True) fails closed when redaction cannot prove safety. It previously printed raw credit-card numbers whenever pattern detection had not run — any metrics= selection omitting the "patterns" pack, or fast_mode, silently disabled the redaction gate. ColumnProfile.patterns is now Option<Vec<Pattern>> end to end: None means unscanned (samples withheld), [] means scanned and clean. (#355, #359)
  • Python: dict input whose columns have differing lengths raises ValueError instead of panicking. Row-dicts with differing keys remain accepted, taking the union of keys with nulls for the missing entries.
  • Python: the database helpers (analyze_database_async, test_connection_async, get_table_schema_async, count_table_rows_async) are exported from the dataprof package. They were documented as dp.<fn> but only ever existed on the private dataprof._dataprof module, so every documented call raised AttributeError. analyze_database_async() now returns a wrapped ProfileReport rather than the raw core report, so report.rows and the other wrapper accessors work as documented. Without a database feature build the names resolve to stubs that raise ImportError explaining the rebuild.
  • Database: decode non-text columns instead of reporting them as all-null. The connectors read every column as Option<String> and dropped the decode error with .ok(), so an INTEGER or REAL value was indistinguishable from SQL NULL and every non-text column profiled as an all-null string on all three connectors. Integer, float, and boolean columns now decode correctly, and integral floats keep their decimal point so a REAL column is still inferred as a float. NUMERIC/DECIMAL, date/time types, and BLOB remain unsupported and still profile as nulls. Present since 0.8.0. Regenerate any quality baseline taken from a database query.
  • Python: correct the type stubs for the database helpers. analyze_database_async takes batch_size/calculate_quality (not a config), get_table_schema_async returns list[str] (not SchemaResult), and count_table_rows_async returns int (not RowCountEstimate).
  • Resolve two clippy lints introduced by Rust 1.97 (question_mark in dataprof-core, byte_char_slices in dataprof-csv).
  • Bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204. (#346)
  • Bump arrow and parquet from 57.3.0 to 58.x. (#324)

Security

  • Upgrade pyo3 and pyo3-async-runtimes from 0.27 to 0.29, clearing a high severity out-of-bounds read in the nth/nth_back iterator implementations for PyList and PyTuple (GHSA-36hh-v3qg-5jq4) and a missing Sync bound on PyCFunction::new_closure (GHSA-chgr-c6px-7xpp). dataprof never called the affected iterator methods, so released versions were not exploitable through the public API, but the vulnerable code was compiled into published wheels.
  • Arrow PyCapsule import now validates capsule names (arrow_schema, arrow_array) before dereferencing the pointers, replacing pyo3's removed unchecked PyCapsule::pointer() accessor with pointer_checked(). A capsule carrying an unexpected payload is now rejected instead of reinterpreted.
  • Bump quinn-proto to 0.11.15 for RUSTSEC-2026-0185.

Internal

  • Python: adopt ty for static type checking, wired into CI as an advisory (non-blocking) step while it is pre-1.0.
  • Centralize crate metadata and dependencies via [workspace.package] and [workspace.dependencies].
  • CI: smoke-install test for the Python wheel and Rust examples, as the 0.9.0 definition of done. (#328)

[0.8.1] - 2026-05-31

Added

  • Python: accept dict, list-of-dicts, bytes, and BytesIO inputs in dp.profile() for notebook-friendly profiling.
  • Python: add dogfooding coverage for sync, async, byte-stream, and local remote-URL profiling with tracked CSV/JSONL fixtures.

Fixed

  • Python: ProfileReport.__getitem__ now raises TypeError for non-string keys instead of KeyError(0).
  • Python: missing file paths now raise standard FileNotFoundError from the Python boundary for profile(), infer_schema(), and quick_row_count().
  • Python: ColumnProfile and DataQualityMetrics now have useful repr() output for notebook inspection.

[0.8.0] - 2026-05-24

Added

  • Add dataprof-engines for adaptive, incremental, and async streaming engine ownership behind the compact dataprof facade.
  • Add dataprof-partial for schema inference and quick row-count ownership behind the compact dataprof facade.
  • Add facade feature-combination checks for no-default and async-streaming builds in CI.
  • Per-column outlier_count (Tukey IQR rule, same k=1.5 as the global accuracy.outlier_ratio) on NumericStats / ColumnProfile to make outlier triage actionable at the column level instead of only the aggregate.
  • QualityMetrics.low_sample_warning (also surfaced as ProfileReport.low_sample_warning in Python) flags reports built from fewer than 10 rows so consumers can decide whether to trust the score.
  • Python: dataprof.column_to_dict(col) returns the same nested dict shape as one element of report.to_dict()["columns"], for piping individual columns into downstream systems without serializing the whole report.
  • Add explicit positive_columns and identifier_columns hints across the Rust and Python profilers. Positive columns drive accuracy.negative_values_in_positive; identifier columns are reported as identifier and excluded from numeric stats/outlier metrics (#304, #305).

Changed

  • Profiler::format() override now wins over extension-based detection on the auto engine path (previously a .parquet filename would route to the Parquet parser even when FileFormat::Csv was forced). Adds the AdaptiveProfiler::analyze_csv_file() entry that bypasses Parquet redetection for callers that have already resolved the format.

  • Remote URL profiling (Profiler::profile_url) validates the URL scheme up-front: malformed inputs like not-a-url now return a clear "expected http(s)://" error instead of reqwest's opaque "builder error".

  • Library noise: small-sample warnings and CSV recovery messages moved from eprintln! to log::warn! so library users opt in via env_logger.

  • Dropped lingering --chunk-size CLI-flag advice from StreamingError and ValidationError (CLI was retired); messages now reference the builder API.

  • Boolean inference now treats true/false case-insensitively and ignores empty/null/nan tokens; yes/no remain strings by default (#303).

  • Move the public dataprof facade source to crates/dataprof/src and remove the legacy src/ shell before the 0.8.0 release.

  • Drop broad legacy module-path shims from the facade; common APIs remain available as top-level exports, while lower-level modules are available from their owning workspace crates.

  • Move async HTTP Parquet profiling into dataprof-parquet.

  • Prepare the workspace for the 0.8.0 Rust facade and Python package release.

  • Refresh release-facing docs around the compact facade, retired CLI surface, and current crate-boundary status.

[0.5.10] - 2026-02-01

Added

  • Add profile_arrow function for PyArrow Table and RecordBatch profiling (#199) by @AndreaBozzo

  • Implement Arrow PyCapsule interface for zero-copy data exchange (#196) by @AndreaBozzo

Changed

  • Autofix n.26 (#198) by @AndreaBozzo

  • Removed legacy req build txt and moved social docs to docs/ folder by @AndreaBozzo

  • Pages: Add workflow to deploy benchmark reports to GitHub Pages (#195) by @AndreaBozzo

Dependencies

  • Deps: Bump thiserror from 2.0.17 to 2.0.18 (#184) by @dependabot[bot]

Documentation

  • Update readme by @AndreaBozzo

  • Logo width 800 by @AndreaBozzo

Fixed

  • Html: Simplify compression ratio calculation in report context for clippy by @AndreaBozzo

  • Build: Add anyhow to dev-dependencies, revert crate-type to rlib only by @AndreaBozzo

[0.5.0] - 2026-01-18

Added

  • Add futures dependency and enhance DataFusionLoader with streaming query profiling by @AndreaBozzo

  • Integrate DataFusion SQL (#177) by @AndreaBozzo

  • Logging: Integrate env_logger for improved logging throughout the application ( bye emojis ) by @AndreaBozzo

  • CI: Add code coverage workflow and remove tarpaulin configuration (testing, might revert) by @AndreaBozzo

  • Enhance performance dashboard with interactive Chart.js visualizations and improved data handling by @AndreaBozzo

  • Add linting configuration and improve chunk processing readability by @AndreaBozzo

  • Refactor chunk processing to use ByteRecord and add batch processing with Rayon for improved performance by @AndreaBozzo

  • Add interactive demo section to README and include demo animation by @AndreaBozzo

Changed

  • Replace info! macro with log::info for consistency in logging by @AndreaBozzo

  • Remove clippy ( present in CI) and fine-tuning by @AndreaBozzo

  • Increase timeout for supply-chain job to 45 minutes by @AndreaBozzo

  • Remove cargo audit configuration and add deny.toml template by @AndreaBozzo

  • Update CSV analysis functions to support optional engine parameter by @AndreaBozzo

  • Remove API reference and integrations documentation for DataProf by @AndreaBozzo

  • Remove outdated guides documentation by @AndreaBozzo

  • Update version to 0.5.0 and adjust dependencies in Cargo files by @AndreaBozzo

  • Fmt by @AndreaBozzo

  • Removed example files and remade security workflow by @AndreaBozzo

  • Remove integration tests from CI workflow matrix by @AndreaBozzo

  • Remove setup of system dependencies from CI workflow by @AndreaBozzo

  • Re-add simplified issue templates for bug reports and feature requests ( same as Ceres ) by @AndreaBozzo

  • Update comments and add feature flags for arrow in integration regression tests by @AndreaBozzo

  • Rename IsoQualityThresholds to IsoQualityConfig for consistency by @AndreaBozzo

  • Rename 'with_*' methods to 'seed' and 'limits' for clarity by @AndreaBozzo

  • Standardize database analysis naming (profile_* → analyze_*) by @AndreaBozzo

  • Add *.so to .gitignore to exclude shared object files by @AndreaBozzo

  • Reorganize version import in Python bindings by @AndreaBozzo

  • Remove SimpleColumnarProfiler and update integration tests to use ArrowProfiler by @AndreaBozzo

  • Remove deprecated development container files and update references to docker-compose by @AndreaBozzo

  • Improvedevelopment databases and testing scripts by @AndreaBozzo

  • Metrics: Split metrics into separate dimensions by @AndreaBozzo

  • Remove unused streaming arguments and related tests for analyze and report commands by @AndreaBozzo

  • Update toml dependency to version 0.9.8 by @AndreaBozzo

  • Clippy & fmt by @AndreaBozzo

  • Tests: Remove outdated testing modules and legacy tests by @AndreaBozzo

  • Replace println! with log statements for improved logging consistency, followup of 7c5a802 by @AndreaBozzo

  • Docs, CI: Add Codecov token to coverage upload and update README with Codecov badge by @AndreaBozzo

  • Add sparse protocol configuration for crates.io by @AndreaBozzo

  • Removed redundant docstrings by @AndreaBozzo

  • Remove pre-commit configuration file by @AndreaBozzo

  • More docs cleanup, this folder will be phased out when i finish transitioning to testcontainers by @AndreaBozzo

  • Removed unused bench folder by @AndreaBozzo

  • Remove outdated README.md for GitHub Actions workflows by @AndreaBozzo

  • Update license badge to reflect dual-license under MIT and Apache 2.0 by @AndreaBozzo

  • Update license information to dual-license under MIT and Apache 2.0; add LICENSE-APACHE file by @AndreaBozzo

  • Update changelog to reflect correct project name and removed those stupid emojis from workflow, this will be an ongoing effort by @AndreaBozzo

  • Remove VSCode snippets and workspace configuration files by @AndreaBozzo

  • Renamed old test file by @AndreaBozzo

  • Remove link to full contributing guide from README by @AndreaBozzo

  • Streamline error handling and row processing in database connectors by @AndreaBozzo

  • Fmt by @AndreaBozzo

  • Update Cargo.toml profile and enhance benchmarks documentation by @AndreaBozzo

Dependencies

  • Deps: Bump trufflesecurity/trufflehog from 3.83.7 to 3.92.4 (#162) by @dependabot[bot]

  • Deps: Bump parquet from 57.1.0 to 57.2.0 (#173) by @dependabot[bot]

  • Deps: Bump arrow from 57.1.0 to 57.2.0 (#172) by @dependabot[bot]

  • Deps: Bump predicates from 2.1.5 to 3.1.3 (#174) by @dependabot[bot]

  • Deps: Bump tokio from 1.48.0 to 1.49.0 (#175) by @dependabot[bot]

  • Deps: Bump toml from 0.9.8 to 0.9.10+spec-1.1.0 (#158) by @dependabot[bot]

Documentation

  • Eliminated redundant doc by @AndreaBozzo

  • Update section title in Code of Conduct to reflect commitment by @AndreaBozzo

Fixed

  • Update CI workflow to skip test suite ( handled by coverages tests, this saves me computing ) on master pushes by @AndreaBozzo

  • Update GitHub Actions workflows to use latest action versions and merge coverage reporting ( no need to run multiple tests runs ) by @AndreaBozzo

  • Update code coverage generation and upload configuration by @AndreaBozzo

  • Update dependencies and improve security audit configuration by @AndreaBozzo

  • Correct capitalization in TruffleHog ignore file header by @AndreaBozzo

  • Update copyright year in LICENSE file to 2026 by @AndreaBozzo

  • Reorder imports to include MemoryMappedCsvReader for memory mapping tests by @AndreaBozzo

  • Remove unnecessary Claude files from .gitignore and add mypy cache by @AndreaBozzo

  • Add "analyze" argument to subprocess calls in benchmark scripts ( yikes ) by @AndreaBozzo

  • Adjust placeholder replacement order in performance dashboard generation script by @AndreaBozzo

  • Update report links to use the correct directory structure and add timestamp generation in benchmarks workflow by @AndreaBozzo

  • Enable LTO in release profile and adjust crate-type for library configuration by @AndreaBozzo

  • Comment out lto in release profile to prevent linking issues with mixed rlib/cdylib by @AndreaBozzo

  • Optimize release profile settings for improved performance, removed useless docstrings by @AndreaBozzo

[0.4.85] - 2026-01-09

Added

  • Overhaul HTML UI, fix Python bindings, and update docs (#168) by @AndreaBozzo

Changed

  • Prepare release 0.4.85 with git-cliff integration by @AndreaBozzo

  • Rename fields in JSON context for clarity in batch report by @AndreaBozzo

Dependencies

  • Deps: Bump tempfile from 3.23.0 to 3.24.0 (#161) by @dependabot[bot]

  • Deps: Bump rsa in the cargo group across 1 directory (#167) by @dependabot[bot]

  • Deps: Bump handlebars from 6.3.2 to 6.4.0 (#166) by @dependabot[bot]

Fixed

  • Add unsafe to Windows extern block and exclude large files from crate by @AndreaBozzo

  • Ci: Use git-cliff action correctly with outputs by @AndreaBozzo

  • Ci: Use git-cliff instead of git cliff (correct command) by @AndreaBozzo

  • Ci: Add debug output for git-cliff and use --current flag by @AndreaBozzo

  • Rust 2024 unsafe extern block and update Cargo.lock by @AndreaBozzo

  • Remove unnecessary panic setting in release profile by @AndreaBozzo

  • Update dataprofhtml2026.png image asset cropping by @AndreaBozzo

[0.4.84] - 2026-01-02

Changed

  • Bump version to 0.4.84 and update changelog by @AndreaBozzo

  • Ease external PR contribution workflow (#165) by @AndreaBozzo

Dependencies

  • Deps: Bump actions/upload-artifact from 5 to 6 (#163) by @dependabot[bot]

  • Deps: Bump actions/download-artifact from 6 to 7 (#164) by @dependabot[bot]

  • Deps: Bump parquet from 57.0.0 to 57.1.0 by @dependabot[bot]

  • Deps: Bump log from 0.4.28 to 0.4.29 by @dependabot[bot]

  • Deps: Bump criterion from 0.7.0 to 0.8.1 by @dependabot[bot]

  • Deps: Bump arrow from 57.0.0 to 57.1.0 by @dependabot[bot]

  • Deps: Bump wide from 0.8.3 to 1.1.0 by @dependabot[bot]

  • Deps: Bump actions/upload-artifact from 4 to 5 by @dependabot[bot]

  • Deps: Bump actions/checkout from 5 to 6 by @dependabot[bot]

  • Deps: Bump pyo3 from 0.27.1 to 0.27.2 by @dependabot[bot]

Documentation

  • Remove batch report animation from README by @AndreaBozzo

  • Update logo size in README for better visibility by @AndreaBozzo

  • Update README with new logo and improved project description by @AndreaBozzo

  • Add enhanced statistical analysis features and remove CI/CD section by @AndreaBozzo

Fixed

  • Ci: Replace deprecated macos-13 with macos-15-intel by @AndreaBozzo

[0.4.83] - 2025-11-28

Added

  • Add precision control for numeric output formatting by @AndreaBozzo

  • Implement enhanced statistical analysis (issue #139) by @AndreaBozzo

Changed

  • Update Cargo.lock for version 0.4.83 by @AndreaBozzo

  • Bump version to 0.4.83 and update changelog by @AndreaBozzo

Dependencies

  • Deps: Bump clap from 4.5.51 to 4.5.53 (#142) by @dependabot[bot]

  • Deps: Bump wide from 0.8.2 to 0.8.3 (#141) by @dependabot[bot]

  • Deps: Bump indicatif from 0.18.2 to 0.18.3 (#140) by @dependabot[bot]

Fixed

  • Update release title to use lowercase 'dataprof' and remove generated comment by @AndreaBozzo

  • Remove unnecessary blank line in quartiles module by @AndreaBozzo

  • Address Copilot review comments by @AndreaBozzo

  • Pre-PR improvements for code quality and compatibility by @AndreaBozzo

[0.4.82] - 2025-11-21

Added

  • Enhance pattern detection with 11 new patterns and performance improvements by @AndreaBozzo

Changed

  • Update version to 0.4.82 and add release notes in changelog by @AndreaBozzo

  • Changelog: Update dependencies section with GitHub Actions and Rust updates and removed outdated notebooks note by @AndreaBozzo

  • Improve clarity in staging to master workflow comments and messages by @AndreaBozzo

Dependencies

  • Deps: Bump actions/upload-pages-artifact from 3 to 4 (#130) by @dependabot[bot]

  • Deps: Bump actions/download-artifact from 4 to 6 (#129) by @dependabot[bot]

  • Deps: Bump wide from 0.8.1 to 0.8.2 (#135) by @dependabot[bot]

  • Deps: Bump github/codeql-action from 3 to 4 (#131) by @dependabot[bot]

Fixed

  • Disable on push and pull_request triggers in staging workflow by @AndreaBozzo

[0.4.81] - 2025-11-15

Added

  • Add code coverage workflow and configuration by @AndreaBozzo

Changed

  • Minor fixes from project review by @AndreaBozzo

Fixed

  • Standardize color value to 'Auto' in tarpaulin configuration by @AndreaBozzo

  • Update timeout values to include 's' suffix for consistency by @AndreaBozzo

  • Update tarpaulin configuration for coverage outputs and simplify coverage command by @AndreaBozzo

[0.4.80] - 2025-11-07

Added

  • Async: Re-enable and improve async support (issue #133) by @AndreaBozzo

Changed

  • Re-added cargo.lock by @AndreaBozzo

  • Bump version to 0.4.80 and update changelog by @AndreaBozzo

  • Applied fmt by @AndreaBozzo

  • Upgrade major dependencies and migrate to modern APIs by @AndreaBozzo

[0.4.78] - 2025-10-23

Changed

  • Bump version to 0.4.78 by @AndreaBozzo

Documentation

  • Fixed typo in readme.md by @AndreaBozzo

Fixed

  • Enable PyO3 forward compatibility for Python 3.14+ by @AndreaBozzo

  • Apply Copilot PR suggestions by @AndreaBozzo

[0.4.77] - 2025-10-16

Added

  • Update version to 0.4.77 - added lazy static - pre-comp regex for better performances - removed some unwrap calls by @AndreaBozzo

Changed

  • Trimmed snippets and ws config couse my setup is slow by @AndreaBozzo

Documentation

  • Update CHANGELOG for v0.4.77 by @AndreaBozzo

  • Update documentation for v0.4.75 with Parquet batch support by @AndreaBozzo

Fixed

  • Apply valid Copilot suggestions by @AndreaBozzo

  • Apply Copilot suggestions - IT phone regex and total_cmp by @AndreaBozzo

  • Clippy fix by @AndreaBozzo

[0.4.75] - 2025-10-09

Added

  • Python: Complete feature parity with Rust core for batch Parquet support by @AndreaBozzo

  • Parquet: Production-ready implementation with extended type coverage by @AndreaBozzo

  • Parquet: Add Apache Parquet format support with unified API integration by @AndreaBozzo

  • Add JSON batch export functionality and clean up formatters by @AndreaBozzo

Changed

  • Update Cargo keywords for crates.io compliance by @AndreaBozzo

  • Post-first-month cleanup - Phase 1 & 2 complete by @AndreaBozzo

  • Re-organized examples directory by @AndreaBozzo

  • Config: Technical debt cleanup - consolidate structures by @AndreaBozzo

  • Parsers: Eliminate expensive parser cloning + remove deprecated field by @AndreaBozzo

  • Engines: Consolidate streaming statistics - eliminate duplication by @AndreaBozzo

  • Core: Major architectural improvements - Priority High by @AndreaBozzo

Documentation

  • Release version 0.4.75 by @AndreaBozzo

  • Update package description to highlight ISO quality metrics by @AndreaBozzo

Fixed

  • Parquet: Resolve conditional compilation for is_parquet_file by @AndreaBozzo

  • Parquet: Resolve feature gate compilation error in core_logic by @AndreaBozzo

  • More imports fixes by @AndreaBozzo

  • Fixed 2 missing imports for CI by @AndreaBozzo

[0.4.70] - 2025-10-03

Added

  • Add batch ML/HTML support and fix quality score calculation by @AndreaBozzo

  • Complete CLI subcommands refactor with backward compatibility (Phase 4/4) ✅ by @AndreaBozzo

  • Add ISO 8000/25012 configurable quality metrics and CLI refactor design by @AndreaBozzo

  • Unify database mode to use full MlReadinessEngine by @AndreaBozzo

  • Enhanced ML readiness analysis with advanced feature intelligence by @AndreaBozzo

  • Complete PyDataQualityMetrics integration and database ML pipeline by @AndreaBozzo

  • Complete DataQualityMetrics display integration by @AndreaBozzo

  • Expose ML features and output options in Python batch functions by @AndreaBozzo

  • Complete batch ML processing integration + critical ML score fix by @AndreaBozzo

  • Enable batch processing functionality + CLI validation fix by @AndreaBozzo

  • Enhanced User Experience & Terminal Intelligence - Issue #79 by @AndreaBozzo

Changed

  • Completed html refactor, added premade templates and a new gif and a dedicated metric test by @AndreaBozzo

  • Complete cleanup - remove all legacy quality code by @AndreaBozzo

  • Migrate HTML generation to Handlebars template engine (WIP) by @AndreaBozzo

  • Massive architecture cleanup - eliminate tech debt (~730 lines) by @AndreaBozzo

  • Eliminate code duplication and fix all clippy warnings by @AndreaBozzo

  • Remove legacy code and simplify architecture (~2400 lines removed) by @AndreaBozzo

  • Add shared core logic for CLI commands to inherit improvements by @AndreaBozzo

  • Cleanup legacy code and enhance transparency documentation by @AndreaBozzo

Documentation

  • Docs update round before 0.4.70 release by @AndreaBozzo

  • Update CHANGELOG with unified database ML implementation by @AndreaBozzo

Fixed

  • Fix ISO compliance tests by meeting minimum sample size requirements by @AndreaBozzo

  • Ignore legacy integration tests with outdated expectations by @AndreaBozzo

  • Fix VSCode config, separate heavy DB tests, rewrite CLI tests for new subcommand structure by @AndreaBozzo

  • I'm out of tears for today pls pass by @AndreaBozzo

  • Complete CI failures - align with QualityReport refactor by @AndreaBozzo

  • Critical HTML quality score bug + complete Timeliness support by @AndreaBozzo

  • Complete QualityIssue removal - establish single source of truth by @AndreaBozzo

  • Critical bugfix - quality score calculation (outlier_ratio & stale_data_ratio) by @AndreaBozzo

  • Complete Python bindings with timeliness metrics (ISO 8000-8) by @AndreaBozzo

  • Calculate actual total_rows and quality metrics in streaming profiler by @AndreaBozzo

  • Update tests for timeliness metrics (ISO 8000-8) by @AndreaBozzo

  • Correct all command implementations with proper types (Phase 3/4) by @AndreaBozzo

  • Clippy warnings in script_generator by @AndreaBozzo

  • Benchmakr comp and html reports for single files, batch and db tests to come after by @AndreaBozzo

  • Add clippy allow for too_many_arguments in Python batch functions by @AndreaBozzo

  • Fixed both BatchConfig initializations in the Python batch processing module by adding the missing required fields by @AndreaBozzo

  • Improve ML readiness scoring algorithm for realistic assessments by @AndreaBozzo

  • Enhanced delimiter detection + complete test suite fixes by @AndreaBozzo

  • CRITICAL - Resolve delimiter detection bug and enhance documentation by @AndreaBozzo

  • Prevent benchmark workflow from running on feature branch pushes by @AndreaBozzo

  • Add missing return value in Linux memory detection by @AndreaBozzo

  • Resolve benchmark compilation errors by @AndreaBozzo

  • Resolve compilation errors across test files and database connectors by @AndreaBozzo

[0.4.61] - 2025-09-27

Added

  • Migrate to dual MIT OR Apache-2.0 licensing by @AndreaBozzo

Changed

  • Bump version to 0.4.61 by @AndreaBozzo

  • Simplify to MIT License only by @AndreaBozzo

Documentation

  • Reorganize README with CI/CD integration and improved CLI documentation by @AndreaBozzo

[0.4.6] - 2025-09-26

Added

  • Optimize CI/CD workflows for 30-40% performance improvement by @AndreaBozzo

  • Add comprehensive demo notebook for v0.4.6 code snippet features by @AndreaBozzo

  • Enhanced ML Recommendations with Actionable Code Snippets - Issue #71 by @AndreaBozzo

  • Complete Phase 3 & 4 development environment standardization by @AndreaBozzo

  • Devenv: Implement standardized development environment (Phase 1 & 2) by @AndreaBozzo

  • Add workflow_dispatch trigger to quick benchmarks by @AndreaBozzo

  • Implement comprehensive statistical rigor framework (#60) by @AndreaBozzo

  • Consolidate and modernize benchmarking infrastructure by @AndreaBozzo

Changed

  • Create concise professional README under 80 lines by @AndreaBozzo

  • Split benchmarks into parallel jobs for better CI performance by @AndreaBozzo

  • Optimize CI workflows for GitHub standard runners by @AndreaBozzo

  • Bump version to 0.4.6 with security enhancements by @AndreaBozzo

  • Major CI/CD workflow optimization with composite actions by @AndreaBozzo

  • Deleted justfile as i'm not using it by @AndreaBozzo

  • Modularize database security.rs for better maintainability by @AndreaBozzo

  • Modularize python.rs into organized module structure by @AndreaBozzo

  • Modularize main.rs into organized CLI structure by @AndreaBozzo

Documentation

  • Fix documentation links in README on master by @AndreaBozzo

  • Fix documentation links in README by @AndreaBozzo

  • Update workflows README for optimized architecture by @AndreaBozzo

  • Add comprehensive benchmarking system documentation by @AndreaBozzo

  • Update changelog with CI/CD improvements by @AndreaBozzo

  • Moved guides toa proper docs/ folder by @AndreaBozzo

  • Update changelog with refactoring improvements by @AndreaBozzo

  • Removed duplicate wiki files by @AndreaBozzo

Fixed

  • Remove empty PYO3_CROSS_LIB_DIR causing macOS build failure by @AndreaBozzo

  • Update CHANGELOG.md version 0.4.6 from Unreleased status by @AndreaBozzo

  • Correct integration test execution in staging-dev workflow by @AndreaBozzo

  • Correct matrix variable access in CI workflow by @AndreaBozzo

  • Replace deprecated crates extension with dependi by @AndreaBozzo

  • Resolve clippy warnings in script generator by @AndreaBozzo

  • Update demo notebook and type stubs for v0.4.6 code snippets by @AndreaBozzo

  • Make setup scripts safer and remove problematic tool installations by @AndreaBozzo

  • Refined .vscode workspace with better rust analyzer config and removed problematic powershell dev script for now by @AndreaBozzo

  • Re-enabled CI checks on staging PRs couse i'm paranoid by @AndreaBozzo

  • Make setup scripts safer and remove problematic tool installations by @AndreaBozzo

  • Eliminate justfile completely and migrate to standard cargo commands by @AndreaBozzo

  • Trimmed pre-commit to only feature code quality checks, while we shift the testing to the gh runners by @AndreaBozzo

  • Show only real data in dashboard, remove fake placeholders by @AndreaBozzo

  • Replace broken sed with perl for dashboard placeholder replacement by @AndreaBozzo

  • Remove malformed Python multiline from YAML workflow by @AndreaBozzo

  • Create functional dashboard with real benchmark data by @AndreaBozzo

  • Replace custom benchmark actions with direct commands by @AndreaBozzo

  • Resolve security workflow issues by @AndreaBozzo

  • Update advanced security workflow to resolve CI failures by @AndreaBozzo

  • Update TruffleHog configuration for proper regex patterns by @AndreaBozzo

  • Resolve clippy warnings and update CI for deprecated PyO3 APIs by @AndreaBozzo

  • Remove system file caching to prevent permission warnings by @AndreaBozzo

  • Correct shell syntax in composite actions by @AndreaBozzo

  • Increase sample-size to meet Criterion minimum requirement by @AndreaBozzo

  • Resolve CI network issues and optimize quick benchmarks by @AndreaBozzo

  • Resolve Criterion sample size assertion failure in statistical benchmarks by @AndreaBozzo

  • Resolve failing test suite issues by @AndreaBozzo

  • Replace unsafe unwrap() calls with safe alternatives by @AndreaBozzo

  • Resolve all clippy errors and improve engine architecture by @AndreaBozzo

  • Resolve clippy compilation errors in engine benchmarks by @AndreaBozzo

  • Add missing Basic pattern support to streaming benchmarks by @AndreaBozzo

  • Resolve domain pattern matching issue with proper module imports by @AndreaBozzo

  • Resolve enum pattern matching issue in streaming domain by @AndreaBozzo

  • Improve pattern matching in streaming domain generation by @AndreaBozzo

  • Resolve unwrap() calls and benchmark compilation issues by @AndreaBozzo

  • Added new justfile by @AndreaBozzo

[0.4.53] - 2025-09-20

Changed

  • Fix Cargo.lock line endings and version by @AndreaBozzo

  • Update dependencies and gitignore by @AndreaBozzo

  • Update Cargo.lock for version 0.4.5 by @AndreaBozzo

Fixed

  • PyPI wheel naming compatibility by @AndreaBozzo

  • Ensure RUSTFLAGS apply to all target architectures by @AndreaBozzo

  • Resolve clippy dead_code warning by @AndreaBozzo

  • Resolve CPU compatibility issues with Python wheels by @AndreaBozzo

[0.4.5] - 2025-09-19

Added

  • Enhance GitHub Pages dashboard and benchmark workflows by @AndreaBozzo

Changed

  • Bump version to 0.4.5 for security release by @AndreaBozzo

  • Enhance GitHub Pages dashboard visual design by @AndreaBozzo

Documentation

  • Adjusted version for changelog.md by @AndreaBozzo

  • Added new Wiki pages by @AndreaBozzo

Fixed

  • Resolve CI test failures and feature compilation issues by @AndreaBozzo

  • Remove unsafe unwrap() call in security module by @AndreaBozzo

  • Address critical SQL injection and information disclosure vulnerabilities by @AndreaBozzo

[0.4.4] - 2025-09-19

Added

  • Complete Python bindings enhancement for ML/AI workflows (#40) by @AndreaBozzo

  • Implement comprehensive CLI enhancements for GitHub issue #39 by @AndreaBozzo

  • Implement comprehensive CLI improvements with ML readiness scoring by @AndreaBozzo

  • Optimize Cargo.toml build profiles and fix unwrap calls for faster compilation by @AndreaBozzo

  • Add comprehensive database enhancements with ML readiness assessment by @AndreaBozzo

Changed

  • Update GitHub Actions workflows and add CLI integration tests by @AndreaBozzo

Documentation

  • Update CHANGELOG.md with Python bindings ML/AI enhancements by @AndreaBozzo

  • Restructure Python documentation with organized wiki structure by @AndreaBozzo

  • Removed markdownlint.yaml as is not being utilized. by @AndreaBozzo

  • Added CLI guide wiki page by @AndreaBozzo

  • Updated CHANGELOG.md to include last 2 PRs merged. by @AndreaBozzo

Fixed

  • Improve security scanner regex for more accurate detection by @AndreaBozzo

  • Remove secret keyword from test to pass security scanner by @AndreaBozzo

  • Improve security test to avoid scanner warnings by @AndreaBozzo

  • Resolve security scanner warnings in test passwords by @AndreaBozzo

  • Replace hardcoded test password with generic placeholder by @AndreaBozzo

  • Remove broken dataprof-wiki submodule by @AndreaBozzo

  • Re-applied linting by @AndreaBozzo

  • Resolve panic strategy conflict for benchmarks by @AndreaBozzo

  • Add missing public exports for CLI binary linking by @AndreaBozzo

  • Restore original thiserror v2.0 configuration by @AndreaBozzo

  • Downgrade thiserror to v1.0 for better compatibility by @AndreaBozzo

  • Replace unsafe unwrap() calls with safe pattern matching by @AndreaBozzo

  • Use is_multiple_of() instead of manual modulo check by @AndreaBozzo

  • Applied formatting to python.rs by @AndreaBozzo

  • Improve git operations resilience in release workflow by @AndreaBozzo

  • Remove all unwrap() calls from progress.rs for safer error handling by @AndreaBozzo

  • Complete CLI enhancements and resolve all test failures by @AndreaBozzo

  • Resolve linking conflicts by using rlib crate type only by @AndreaBozzo

  • Remove problematic build rustflags that caused linking errors by @AndreaBozzo

  • Optimize staging-dev workflow to use default database features by @AndreaBozzo

  • Clarify database features are default, not optional + optimize CI workflows by @AndreaBozzo

  • Update DatabaseConfig initialization in database_examples.rs by @AndreaBozzo

  • Relax performance test threshold to 3.0x for CI environments by @AndreaBozzo

[0.4.1] - 2025-09-16

Added

  • Complete Arrow integration and intelligent engine selection by @AndreaBozzo

  • Validate and refine performance claims with comprehensive benchmarking by @AndreaBozzo

Changed

  • Add Python artifacts to .gitignore by @AndreaBozzo

  • Remove legacy setup.py files by @AndreaBozzo

Documentation

  • Update CLAUDE.md with essential commands by @AndreaBozzo

  • Add v0.4.1 changelog for intelligent engine selection by @AndreaBozzo

Fixed

  • Relax performance test threshold for CI environments by @AndreaBozzo

  • Removed windows path for linux ci runner by @AndreaBozzo

  • Formatting cleanup by @AndreaBozzo

  • Make engine selection tests deterministic by @AndreaBozzo

  • Improve engine selection test for large files by @AndreaBozzo

  • Resolve cross-platform binary path issue in benchmark script by @AndreaBozzo

  • Improve GitHub Pages dashboard template and update CHANGELOG by @AndreaBozzo

  • Add fallback return in memory usage detection for Linux by @AndreaBozzo

[0.4.0] - 2025-09-14

Added

  • Implement comprehensive performance benchmarking suite - resolves #23 by @AndreaBozzo

  • Improve release workflow with comprehensive automation by @AndreaBozzo

  • Comprehensive test coverage infrastructure - issue #21 by @AndreaBozzo

Changed

  • Bump version to 0.4.0 with comprehensive changelog by @AndreaBozzo

  • Implement modular architecture for lib.rs - resolves issue #20 by @AndreaBozzo

Documentation

  • Update changelog with comprehensive quality assurance features by @AndreaBozzo

  • Redesign README for professional elegance and clarity by @AndreaBozzo

  • Add Apache Arrow integration guide and update CHANGELOG v0.3.6 by @AndreaBozzo

  • Add Code of Conduct for community guidelines by @AndreaBozzo

Fixed

  • Eliminate environment variable dependency for binary upload by @AndreaBozzo

  • Add shell bash for wheel upload on Windows by @AndreaBozzo

  • Improve binary preparation for cross-platform release by @AndreaBozzo

  • Resolve GitHub Pages 404 for benchmark results by @AndreaBozzo

  • Update GitHub Pages deployment to use actions/deploy-pages@v4 by @AndreaBozzo

  • Updated benchmarks artifact generation on a specifical github page by @AndreaBozzo

  • Update benchmark workflow triggers and criterion output format by @AndreaBozzo

  • Use standard GitHub Actions runners instead of custom labels by @AndreaBozzo

  • Resolve criterion sample-size requirement and workflow skipping behavior by @AndreaBozzo

[0.3.6] - 2025-09-11

Added

  • Replace broken workflows with simple, human-logic release workflow by @AndreaBozzo

Fixed

  • Add --find-interpreter flag to maturin build by @AndreaBozzo

  • Correct matrix strategy syntax in release workflow by @AndreaBozzo

[0.3.6.1] - 2025-09-11

Changed

  • Update Cargo.lock for version 0.3.6 [skip ci] by @AndreaBozzo

  • Release: Bump version to 0.3.6 [skip ci] by @AndreaBozzo

  • Release: V0.3.6 [skip ci] by @AndreaBozzo

  • Update gitignore for Node.js dependencies by @AndreaBozzo

Dependencies

  • Deps: Bump thiserror from 1.0.69 to 2.0.16 (#14) by @dependabot[bot]

Fixed

  • Resolve duplicate thiserror entries in Cargo.lock by @AndreaBozzo

[0.3.5] - 2025-09-08

Added

  • Enhance release automation and fix crates.io publishing by @AndreaBozzo

  • Optimize memory allocations and add leak detection for v0.3.5 by @AndreaBozzo

  • Add comprehensive database connector system by @AndreaBozzo

  • Setup staging branch development workflow by @AndreaBozzo

Changed

  • Update to v0.3.5 - Database Connectors & Memory Safety Edition by @AndreaBozzo

  • Simplify staging-to-master workflow and add release automation by @AndreaBozzo

  • Add comprehensive memory leak detection tests by @AndreaBozzo

  • Temporarily disable Python features on macOS by @AndreaBozzo

  • Apply code formatting and linter fixes by @AndreaBozzo

Fixed

  • Handle existing versions gracefully in both PyPI and crates.io by @AndreaBozzo

  • Resolve compilation errors and security audit false positive by @AndreaBozzo

  • Support SQLite :memory: connections and clarify sampling key usage by @AndreaBozzo

  • Resolve streaming profiler test failures by @AndreaBozzo

  • Finalize workflows and formatting for v0.3.5 release by @AndreaBozzo

  • Replace unsafe unwrap() calls with proper error handling by @AndreaBozzo

  • Resolve --all-features conflicts and simplify CI workflows by @AndreaBozzo

  • Resolve Clippy warnings and align pre-commit hooks with CI workflows by @AndreaBozzo

  • Resolve DuckDB linking issues in CI by testing features separately by @AndreaBozzo

  • Correct unwrap() detection logic in CI workflow by @AndreaBozzo

  • Update CI workflows for database features compatibility by @AndreaBozzo

  • Remove unsafe unwrap() calls throughout codebase by @AndreaBozzo

  • Use GitHub Actions conditionals instead of bash syntax by @AndreaBozzo

  • Use correct PyO3 0.24+ Bound API pattern by @AndreaBozzo

  • Add explicit PyO3 imports for v0.24+ compatibility by @AndreaBozzo

  • Remove all unsafe unwrap() calls - resolves #18 by @AndreaBozzo

  • Reduce clippy strictness in staging workflow by @AndreaBozzo

Security

  • Upgrade PyO3 to v0.24.2 to fix RUSTSEC-2025-0020 by @AndreaBozzo

[0.3.0] - 2025-09-06

Added

  • Comprehensive v0.3.0 development infrastructure and tooling by @AndreaBozzo

  • Implement v0.3.0 streaming and memory-efficient processing by @AndreaBozzo

Dependencies

  • Deps: Bump colored from 2.2.0 to 3.0.0 by @dependabot[bot]

  • Deps: Bump clap from 4.5.46 to 4.5.47 by @dependabot[bot]

Fixed

  • Resolve all clippy warnings and pre-commit issues by @AndreaBozzo

  • Apply pre-commit formatting and linting fixes by @AndreaBozzo

[0.2.0] - 2025-09-03

Added

  • Implement comprehensive data quality analysis system by @AndreaBozzo

[0.1.0] - 2025-09-02