Skip to content

Latest commit

 

History

History
1620 lines (850 loc) · 93.1 KB

CHANGELOG.md

File metadata and controls

1620 lines (850 loc) · 93.1 KB

Changelog

All changes that impact users of this module are documented in this file, in the Common Changelog format with some additional specifications defined in the CONTRIBUTING file. This codebase adheres to Semantic Versioning.

17.0.2 - 2026-11-18

Development of this release was supported by Anthelia.

Fixed

  • Stop aborting tracking runs with a cannot lock ref 'HEAD' error on servers that run Git 2.47 to 2.50, such as Debian 13, by making the commit that triggers the automatic maintenance of Git wait for it instead of leaving it to a background process; the tracking run pauses meanwhile, usually about a minute on a repository of several gigabytes, and occasionally several minutes when Git consolidates its packs

17.0.1 - 2026-11-10

Development of this release was supported by Anthelia.

Fixed

  • Relaunch the headless browser when it crashes, instead of failing every following fetch of the run on its closed connection; the terms being fetched when it crashes get a new attempt at the end of the tracking run

17.0.0 - 2026-10-14

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Changed

  • Use the OTA_ENGINE_GITHUB_TOKEN environment variable only to publish datasets, as the engine no longer needs it to track terms
  • Publish datasets to GitLab when the OTA_ENGINE_GITLAB_RELEASES_TOKEN environment variable is defined, as it is the token that authenticates the publishing requests; the OTA_ENGINE_GITLAB_TOKEN environment variable is no longer used and can be removed

Removed

  • Breaking: Remove the reporter that created issues on GitHub or GitLab when tracking failed, along with its reporter configuration; install the @opentermsarchive/issue-reporter module in the collection, move the reporter configuration to its @opentermsarchive/issue-reporter key with an additional collectionApi.url entry, replacing any legacy githubIssues entry with "type": "github" next to the same repositories, rename the token environment variable to OTA_ISSUE_REPORTER_GITHUB_TOKEN or OTA_ISSUE_REPORTER_GITLAB_TOKEN, and run ota-issue-reporter sync --schedule alongside the engine, as described in how to report tracking failures
  • Breaking: Remove the scripts/reporter/duplicate script, which closed the duplicate issues that engines older than v2.3.2 could create

Fixed

  • Publish datasets to GitLab through the proxy defined by the lowercase http_proxy or https_proxy environment variables, or by HTTP_PROXY alone, as the fetching of terms already does

16.4.0 - 2026-10-07

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add the runId of the completed tracking run reflected by the /tracking-results and /tracking-result/{serviceId} responses of the Collection API, so that consumers paginating through them can tell when a run completed between two pages and start over

16.3.2 - 2026-10-07

Development of this release was supported by Anthelia.

Fixed

  • Create new versions, snapshots and tracking-results repositories on the main branch, the default branch of GitHub and GitLab and the one used by the deployment, instead of the default branch of the Git configuration of the server, often master; this requires Git 2.28 or later; to move an existing repository to main, stop the tracker and run in its local directory:

    git branch -m master main
    git push -u origin main # Add --force if the remote main branch only holds the files created with the repository
    git push origin --delete master # After making main the default branch of the remote repository

16.3.1 - 2026-10-07

Development of this release was supported by Anthelia.

Fixed

  • Fix the links to the declaration, version and snapshot files in the tracking issues of services whose name differs from their ID, which pointed to nonexistent files named after the service name instead of its ID
  • Fix the links to the snapshots of terms combining multiple source documents in the tracking issues, which pointed to nonexistent files

16.3.0 - 2026-09-30

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add Collection API endpoints serving the latest known tracking status of each terms and the summary of the latest completed tracking run: /tracking-results, filterable by status, /tracking-result/{serviceId}, /tracking-result/{serviceId}/{termsType} and /tracking-results/run; the tracking results of terms removed from the declarations are kept and flagged as not declared; these endpoints are not exposed when tracking-results is disabled

16.2.0 - 2026-09-28

Development of this release was supported by Anthelia.

Changed

  • Refuse to start a process that writes to a Git repository already written by another running process, such as a second ota track started while the scheduled tracker runs, instead of letting both race the working tree and the index, which made one of them fail or discard the uncommitted changes of the other; stop the running writer first, the error names it and the .git/ota-writer.lock file that holds the lock until it exits

Fixed

  • Remove at each tracking run the index.lock file left behind by a Git operation interrupted by a kill, which otherwise made the tracker fail at every restart with index.lock: File exists

16.1.1 - 2026-09-23

Development of this release was supported by Anthelia.

Fixed

  • Fix the "Latest version" and "Latest snapshot" links in tracking issues, which pointed into the declarations repository instead of the versions and snapshots repositories

16.1.0 - 2026-09-23

Development of this release was supported by User Rights and by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add a tracking-results data repository recording the tracking status of each terms and the lifecycle of every run in a tamper-evident Git history (RFC)
  • Add @opentermsarchive/engine.tracking-results configuration defining the storage location, author and publication of the tracking-results repository; defaults record locally to ./data/tracking-results without publication; set it to null to disable tracking-results
  • Add an x-run-id metadata to snapshots and versions recorded while a tracking-results run is in progress, tying each record to the run that produced it; it is a commit trailer with Git storage
  • Reject in ota validate the service IDs containing characters that cannot be used in file names on every supported platform (/, \, :, ", <, >, |, *, ?, control characters), as documented in the service ID guidelines

Changed

  • Mention the declared URL instead of the last redirect target in the errors of fetches that fail because of redirects, so that the error stays the same from one tracking to the next

Fixed

  • Record as a technical upgrade the version of combined terms whose newly declared source document could only be fetched on a retry, instead of refetching all their source documents and recording a regular change that notified a false modification of the terms
  • Stop stating in tracking issues that the source documents were recorded as snapshots and that missed versions may be recovered, when they could actually not be fetched
  • Fix the links to the latest snapshots in the tracking issues of combined terms, which pointed to a nonexistent file with a .null extension for every source document that was successfully fetched; also omit these links for source documents that have never been recorded as snapshots, as there is no file to link to
  • Pass falsy parameters, such as false or 0, to collection-specific filters, which received the extraction context in their place
  • Track only once a service whose ID is given several times to ota track --services

16.0.2 - 2026-09-16

Development of this release was supported by Anthelia.

Fixed

  • Stop the Collection API and the dataset export from resetting the working tree and rewriting the commit-graph of the repositories they only read, which raced the tracker started at the same time and made either fail with commit-graph.lock: File exists
  • Read PDF versions and snapshots straight from the Git object database instead of temporarily restoring them in the working tree, which could make a concurrent tracker commit stale content
  • Fail with an explicit error when the Collection API or the dataset export opens a repository that does not exist yet, instead of silently querying the enclosing repository

16.0.1 - 2026-09-16

Development of this release was supported by Anthelia.

Fixed

  • Report a failure to apply the user agent override or to enable PDF interception in the full DOM fetcher as an error on the fetched document instead of exiting the whole tracking process on an unhandled promise rejection
  • Register the full DOM fetcher evasions once per browser launch instead of adding a copy on every tracking run, which multiplied the scripts and commands sent to Chrome for every page

16.0.0 - 2026-09-16

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Store the latest generated dataset locally in the directory defined by the new dataset.storagePath configuration option (defaults to ./data/datasets), alongside a metadata.json file describing it; this directory is managed by the engine, which deletes any .zip archive it does not reference at each generation, so it must not be shared with other files; see the RFC
  • Add GET /dataset/latest and GET /dataset/latest/download endpoints to the Collection API to expose the metadata and the archive of the latest locally stored dataset, with HTTP caching and resumable downloads

Changed

  • Breaking: Write the dataset archive generated by ota dataset into dataset.storagePath instead of the current working directory, atomically, and keep it after publishing; make sure the machine has enough disk space to keep two dataset archives during generation

Removed

  • Breaking: Remove the --remove-local-copy option of ota dataset, as the local copy is now the reference dataset served by the Collection API; remove this option from the dataset:schedule script of the collection package.json

Fixed

  • Fix GitLab dataset publishing failing after the release creation because of a missing debug log level
  • Fix the dataset scheduler exiting after a failed release; the error is now logged and the next scheduled release still runs

15.3.1 - 2026-09-09

Development of this release was made on a volunteer basis by be-student.

Fixed

  • Reject duplicate source document locations in combine during schema-only validation, as full validation already does

15.3.0 - 2026-09-09

Development of this release was supported by Anthelia.

Changed

  • Name the collection in the subject of Collection API error emails and give them the same body as engine error emails
  • Prefix the subject and title of error emails with the environment name, such as [development], when NODE_ENV is not production

Fixed

  • Keep the Collection API running when sending an error email fails
  • Send one error email per unhandled promise rejection instead of two
  • Wait for the error email to be sent before exiting on an unhandled promise rejection
  • Stop sending errors a second time as warnings when logger.sendMailOnError.sendWarnings is enabled
  • Send error emails as HTML instead of plain text
  • Start the engine when logger.sendMailOnError is enabled without sendWarnings
  • Start the Collection API with a warning instead of failing when OTA_ENGINE_SMTP_PASSWORD is missing while logger.sendMailOnError is enabled

15.2.0 - 2026-09-09

Development of this release was made on a volunteer basis by be-student.

Changed

  • Remove well-known tracking query parameters (utm_*, fbclid, gclid, ...) by default when the built-in removeQueryParams filter is declared without parameters; see the built-in filters documentation

15.1.0 - 2026-07-13

Development of this release was supported by User Rights.

Added

  • Add convertSpacesToStandard built-in filter that replaces Unicode space separators (non-breaking space, narrow no-break space, ...) with a regular space; see more in the built-in filters documentation

15.0.0 - 2026-07-07

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add paginated GET /versions, GET /versions/{serviceId} and GET /versions/{serviceId}/{termsType} endpoints to the Collection API to list versions
  • Add GET /version/{versionId} and GET /version/{serviceId}/{termsType}/latest endpoints to the Collection API to retrieve a single version
  • Expose serviceId, termsType, isFirstRecord, isTechnicalUpgrade, source fetchUrls, navigation links and diff statistics (additions and deletions) in Collection API version responses

Changed

  • Breaking: Return the GET /service/{serviceId} not-found response as JSON instead of plain text

Fixed

  • Reject version lookups whose service ID, terms type or record ID would be parsed as a Git option, which previously let an unauthenticated caller overwrite arbitrary files through the Collection API
  • Return 404 instead of a server error when a version lookup contains path separators or relative path segments; the error previously exposed the repository filesystem location
  • Return a generic message instead of internal error details in Collection API 500 responses

14.1.0 - 2026-06-29

Development of this release was supported by the Research Chair in Content Moderation at the Conservatoire National des Arts et Métiers.

Added

  • Validate that no combine lists the same source document location more than once within a terms type

14.0.0 - 2026-06-23

Development of this release was supported by Reset Tech.

Changed

  • Breaking: Update minimum supported Node.js version to v22
  • Update supported Node.js version to v26

Fixed

  • Recover automatically from a stale commit-graph.lock left behind when a commit-graph write is interrupted
  • Prevent the full DOM fetcher from hanging indefinitely when a page navigation exceeds the configured timeout

13.0.1 - 2026-06-23

Development of this release was made on a volunteer basis by Bhavy Singhal.

Fixed

  • Improve error message when history file is wrongly formatted as an object instead of an array

13.0.0 - 2026-06-22

Development of this release was supported by User Rights.

Changed

  • Breaking: Set navigator.language and navigator.languages in the full DOM fetcher strictly from the tags in @opentermsarchive/engine.fetcher.language through the stealth evasions registered at browser launch rather than a per-page CDP override

Removed

  • Breaking: Drop support for quality factors (;q=…) in @opentermsarchive/engine.fetcher.language; previously accepted values such as en-IE,en-GB;q=0.9,en;q=0.8 now throw at launch. Replace with a plain comma-separated priority list, e.g. en-IE,en-GB,en

Fixed

  • Reactivate the puppeteer-extra stealth plugin in the full DOM fetcher, which had been silently inert since v10.3.1 because it was registered after puppeteer.launch(), restoring removal of navigator.webdriver and HeadlessChrome from the user agent

12.0.2 - 2026-05-26

Development of this release was supported by User Rights.

Fixed

  • Update @accordproject/markdown-cicero to fix npm ci failures in collections caused by broken file:.. references in the deprecated transitive @accordproject/concerto-core@3.19.8

12.0.1 - 2026-05-26

Development of this release was supported by User Rights.

Fixed

  • Avoid GitHub API "body is too long" error when reporting issues for declarations using combine, by linking directly to the declaration file on GitHub instead of embedding the full declaration JSON in the contribution tool URL.

12.0.0 - 2026-05-20

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add GET /feed endpoint on the Collection API exposing an Atom feed of the latest version changes across the whole collection
  • Add GET /feed/:serviceId endpoint on the Collection API exposing an Atom feed scoped to a single service
  • Add GET /feed/:serviceId/:termsType endpoint on the Collection API exposing an Atom feed scoped to a single service and terms type
  • Add @opentermsarchive/engine.collection-api.feed.limit configuration option controlling the maximum number of entries returned by feed endpoints (default: 100)
  • Add @opentermsarchive/engine.collection-api.feed.versionUrlTemplate configuration option to customize the alternate link of feed entries with a URL template (e.g. https://github.com/openTermsArchive/demo-versions/commit/%VERSION_ID); useful to point feed readers to a human-readable page instead of the default version API endpoint

Changed

  • Breaking: Resolve serviceId path parameter case-sensitively on the GET /service/:serviceId endpoint, in line with the documented service ID format; clients relying on case-insensitive matching must now use the exact ID casing

11.0.2 - 2026-04-14

Development of this release was supported by Reset Tech.

Fixed

  • Fix snapshot storage failure when source document URLs contain characters forbidden in filenames
  • Fix duplicate file extension in snapshot filenames when source document URLs end with an extension

11.0.1 - 2026-04-06

Development of this release was supported by Reset Tech.

Fixed

  • Update JSDOM to fix extraction of terms with large inline stylesheets

11.0.0 - 2026-03-30

Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Removed

  • Remove Notifier module
  • Breaking: Remove all default SMTP values (host, port, username) from logger configuration; operators enabling sendMailOnError must now explicitly set all three in @opentermsarchive/engine.logger.smtp

10.6.1 - 2026-02-05

Development of this release was supported by Reset Tech.

Fixed

  • Fix declaration validation when a new service is added along with a filters file

10.6.0 - 2026-01-26

Development of this release was supported by Reset Tech.

Changed

  • Treat navigation timeout errors as transient, enabling automatic retry when pages take too long to load

10.5.0 - 2026-01-20

Development of this release was supported by Reset Tech.

Added

  • Add PDF file support in full DOM fetcher

10.4.0 - 2026-01-19

Development of this release was supported by Reset Tech.

Changed

  • Make SMTP port configurable and switch from implicit SSL to STARTTLS, enabling use of alternative ports (like 2525) on servers where standard SMTP ports are blocked

10.3.3 - 2026-01-16

Development of this release was supported by Reset Tech.

Fixed

  • Fix dataset publishing failure for large files by executing uploads sequentially instead of in parallel

10.3.2 - 2026-01-14

Development of this release was supported by Reset Tech.

Fixed

  • Fix error messages when the start and end selectors of range selectors match elements on the page but the selected range or extracted content is empty

10.3.1 - 2026-01-13

Development of this release was supported by Reset Tech.

Fixed

  • Fix browser and request language with full DOM fetcher

10.3.0 - 2026-01-12

Development of this release was made on a volunteer basis by @Ndpnt.

Changed

  • Set realistic viewport size for full DOM fetcher to avoid detection based on default Puppeteer dimensions
  • Improve isolation between fetches by using isolated browser contexts

10.2.0 - 2026-01-08

Full changeset and discussions: #1219.

Development of this release was supported by Reset Tech.

Changed

10.1.3 - 2026-01-06

Full changeset and discussions: #1217.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Validate presence of required configuration for dataset publication to data.gouv.fr

10.1.2 - 2025-12-09

Full changeset and discussions: #1212.

Development of this release was supported by Reset Tech.

Fixed

  • Exclude development and local config files from the published package to avoid unintentionally overriding default configuration

10.1.1 - 2025-11-25

Full changeset and discussions: #1209.

Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.

Fixed

  • Support proxies in GitLab issues reporting

10.1.0 - 2025-11-24

Full changeset and discussions: #1202.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add support for publishing datasets to data.gouv.fr; configure dataset.datagouv.datasetId or dataset.datagouv.organizationIdOrSlug in configuration file and set OTA_ENGINE_DATAGOUV_API_KEY environment variable
  • Add ability to publish datasets to multiple platforms simultaneously; datasets can now be published to GitHub (or GitLab) and data.gouv.fr in parallel

10.0.1 - 2025-11-24

Full changeset and discussions: #1208.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Prevent external hosts from accessing the collection API directly

10.0.0 - 2025-11-20

Full changeset and discussions: #1207.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add ota apply-technical-upgrades CLI command to apply technical upgrades independently

Changed

  • Breaking: Remove --extract-only option from ota track command; use the new ota apply-technical-upgrades command instead

Fixed

  • Fix incorrect versioning that occurred when adding new source documents to combined terms declarations

9.2.3 - 2025-11-19

Full changeset and discussions: #1204.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix email reporting on collection API errors

9.2.2 - 2025-11-19

Full changeset and discussions: #1203.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix saving records whose content is not yet loaded when using MongoDB storage.

9.2.1 - 2025-11-19

Full changeset and discussions: #1206.

Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.

Fixed

  • Fix tracking when proxy configuration is provided

9.2.0 - 2025-11-05

Full changeset and discussions: #1173.

Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.

Added

  • Add proxy support for fetching documents behind firewalls or restricted networks; configure using HTTP_PROXY and HTTPS_PROXY (or http_proxy and https_proxy) environment variables
  • Add debugging options to disable headless mode for visual troubleshooting during development; set OTA_ENGINE_FETCHER_NO_HEADLESS=1 to show browser window
  • Add sandbox control for improved compatibility with Docker and containerized environments; set OTA_ENGINE_FETCHER_NO_SANDBOX=1 when running in containers

9.1.2 - 2025-10-30

Full changeset and discussions: #1199.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Update Node compatibility metadata; actual compatibility was added in v7.2.1

9.1.1 - 2025-10-07

Full changeset and discussions: #1198.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Increase robustness of PDF content type detection

9.1.0 - 2025-10-01

Full changeset and discussions: #1197.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Improve memory management when tracking terms with multiple source documents
  • All available snapshots are now stored in multi-documents terms, even if some are unavailable.

9.0.0 - 2025-09-25

Full changeset and discussions: #1196.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Update collection governance roles by adding contributor, analyst and diffuser and replacing administrator with sysadmin; update roles in your metadata file at the root of your collection folder

8.0.0 - 2025-09-22

Full changeset and discussions: #1193.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Update minimum supported Node.js version to v20
  • Add Content Monetisation Policy terms type, introduced in terms-types@2.1.0
  • Update dependencies

7.2.4 - 2025-09-22

Full changeset and discussions: #1195.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix built-in removeQueryParams filter to not modify URLs without target parameters

7.2.3 - 2025-09-17

Full changeset and discussions: #1192.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix declarations validation when filters history is modified

7.2.2 - 2025-09-16

Full changeset and discussions: #1191.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix validation of declaration files with filters

7.2.1 - 2025-09-11

Full changeset and discussions: #1189.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add support for Node.js v24

7.2.0 - 2025-09-11

Full changeset and discussions: #1178.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

7.1.0 - 2025-09-10

Full changeset and discussions: #1188.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Improve readability of multi-document terms

7.0.0 - 2025-07-21

Full changeset and discussions: #1175.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Update issue labels to improve clarity; existing labels will be automatically updated on the first run

6.1.0 - 2025-07-21

Full changeset and discussions: #1176.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Changed

  • Fetch multiple source documents sequentially to prevent bot detection and improve tracking success rate

6.0.1 - 2025-07-07

Full changeset and discussions: #1171.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Ensure labels are properly recreated after removing deprecated labels in GitHub and GitLab reporters

6.0.0 - 2025-07-01

Full changeset and discussions: #1170.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Replace generic issue labels with more descriptive names to better understand why terms tracking was interrupted; existing labels will be automatically updated on the first run
  • Add ⚠ needs intervention label to highlight critical issues requiring manual action to restore terms tracking

5.7.0 - 2025-06-30

Full changeset and discussions: #1169.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Improve headless browser context isolation when fetching pages by disabling cache and clearing cookies between requests to prevent session persistence across different URLs and to improve tracking success rate

5.6.1 - 2025-06-30

Full changeset and discussions: #1168.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix automatic labeling of GitHub issues when tracking fails due to DNS resolution errors

5.6.0 - 2025-06-19

Full changeset and discussions: #1166.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Added

  • Extend automatic retry mechanism for failed tracking attempts due to likely bot blocking errors to improve tracking success rate

5.5.0 - 2025-06-04

Full changeset and discussions: #1159.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add content validation to ensure selectors contain actual text before considering them loaded to improve tracking success rate

5.4.2 - 2025-06-02

Full changeset and discussions: #1162.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix engine version in logs, records metadata and collection API

5.4.1 - 2025-05-26

Full changeset and discussions: #1156.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix network system error reporting

5.4.0 - 2025-05-21

Full changeset and discussions: #1155.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Improve email error and warning reporting to provide more detailed information and better readability

5.3.1 - 2025-05-20

Full changeset and discussions: #1154.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Handle connection resets as bot protection to improve tracking success rate

5.3.0 - 2025-05-14

Full changeset and discussions: #1152.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Added

  • Add an automatic fetching fallback to headless browser when bot detection is encountered to improve tracking success rate

5.2.0 - 2025-05-13

Full changeset and discussions: #1151.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Added

  • Add an automatic retry mechanism for failed tracking attempts due to likely transient errors to improve tracking success rate

    ⚠️ Version correction notice: Version 5.1.1 was incorrectly published as a patch release, violating semantic versioning principles. This version (5.2.0) contains the same changes but is published as a minor version to properly reflect the addition of the new automatic retry mechanism feature.

5.1.1 - 2025-05-13

Full changeset and discussions: #1150.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Added

  • Add an automatic retry mechanism for failed tracking attempts due to likely transient errors to improve tracking success rate

5.1.0 - 2025-05-12

Full changeset and discussions: #1149.

Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).

Changed

  • Reduce navigation timeout errors and enhance performance of headless browser fetching

5.0.6 - 2025-05-09

Full changeset and discussions: #1148.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Update links to documentation in tracking issues reports

5.0.5 - 2025-05-07

Full changeset and discussions: #1147.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Enable declarations linting in production environments

5.0.4 - 2025-05-06

Full changeset and discussions: #1146.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix creation of issues labels for GitHub repositories with over 30 labels

5.0.3 - 2025-03-10

Full changeset and discussions: #1141.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix MongoDB repository support of terms with multiple source documents

5.0.2 - 2025-03-05

Full changeset and discussions: #1140.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix validation of the collection metadata file

5.0.1 - 2025-03-04

Full changeset and discussions: #1139.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Validate required fields in collection metadata file

5.0.0 - 2025-02-18

Full changeset and discussions: #1137.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Rename documents key to terms in service declarations for vocabulary consistency; to update your declarations:

    For Unix-like systems:

    find . -name "*.json" -type f -exec sed -i 's/"documents":/"terms":/g' {} +

    For macOS:

    find . -name "*.json" -type f -exec sed -i '' 's/"documents":/"terms":/g' {} +

    ⚠️ Note: when updating the engine, CI checks of modified declarations will fail once. This is expected and can be safely ignored

4.2.0 - 2025-02-17

Full changeset and discussions: #1135.

Development of this release was supported by the European Union.

Added

  • Add support for nested subgroups in Gitlab reporter repository paths

Fixed

  • Fix error logging in Gitlab reporter

4.1.0 - 2025-02-17

Full changeset and discussions: #1134.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add ota validate metadata command to the CLI to validate the collection metadata file

4.0.2 - 2025-02-04

Full changeset and discussions: #1133.

Development of this release was supported by the European Union.

Fixed

  • Fix functionality of filtering by status for the issues list on GitLab

4.0.1 - 2024-12-11

Full changeset and discussions: #1124.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix documentation of collection metadata in the collection API

4.0.0 - 2024-12-04

Full changeset and discussions: #1123.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Breaking: Expose collection metadata through the collection API; ensure to add the required metadata file at the root of your collection folder

Changed

  • Breaking: Replace @opentermsarchive/engine.services.declarationsPath with @opentermsarchive/engine.collectionPath; ensure your declarations are located in ./declarations in your collection folder

3.0.0 - 2024-12-03

Full changeset and discussions: #1122.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Updated peer dependency @opentermsarchive/terms-types to v2.0.0; in your declarations, if you define a specific version of this library, ensure to also upgrade it to v2.0.0 to avoid compatibility issues; see terms-types@2.0.0 for details

2.7.2 - 2024-11-27

Full changeset and discussions: #1121.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Optimize performance for Git storage

2.7.1 - 2024-11-21

Full changeset and discussions: #1120.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix timestamp prefix format in logs

2.7.0 - 2024-11-20

Full changeset and discussions: #1118.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

2.6.0 - 2024-11-19

Full changeset and discussions: #1116.

Development of this release was supported by the European Union and the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add support for GitLab for issue reporting
  • Add support for GitLab Releases for publishing datasets

2.5.0 - 2024-10-29

Full changeset and discussions: #1115.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add script to remove duplicate issues in GitHub reports

2.4.0 - 2024-10-24

Full changeset and discussions: #1114.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

2.3.3 - 2024-10-23

Full changeset and discussions: #1113.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Reduce GitHub API calls in issues reporter

2.3.2 - 2024-10-23

Full changeset and discussions: #1112.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Reduce API calls and avoid inconsistent states on GitHub issues reports

2.3.1 - 2024-10-22

Full changeset and discussions: #1111.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Prevent duplicate issues on GitHub reports

2.3.0 - 2024-10-21

Full changeset and discussions: #1110.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Bump @opentermsarchive/terms-types peer dependency to support new types introduced in v1.3.0

2.2.2 - 2024-10-21

Full changeset and discussions: #1109.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Prevent the engine from exiting on SMTP errors from the logger

2.2.1 - 2024-06-07

Full changeset and discussions: #1088.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Display engine version in logs

Fixed

  • Fix the number of terms displayed in logs
  • Fix Reporter module instantiation

2.2.0 - 2024-06-05

Full changeset and discussions: #1087.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Set default tracking frequency to two per day

2.1.0 - 2024-06-04

Full changeset and discussions: #1086.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Make tracking schedule configurable
  • Make publishing schedule configurable
  • Add tracking overrun protection

2.0.1 - 2024-05-31

Full changeset and discussions: #1085.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Corrected a missing negation in a Reporter error log

2.0.0 - 2024-05-22

Full changeset and discussions: #1071.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Nest all configurations under the @opentermsarchive/engine key; wrap the content of config/production.json like this: { @opentermsarchive/engine: { < your previous configuration >}}
  • Breaking: Rename the api configuration key to collection-api; update this in your config/production.json
  • Breaking: Remove default values for port and basePath in collection-api; specify these configurations explicitly in your config/production.json
  • Breaking: Prefix all environment variables with OTA_ENGINE_; rename variables such as SENDINBLUE_API_KEY to OTA_ENGINE_SENDINBLUE_API_KEY, SMTP_PASSWORD to OTA_ENGINE_SMTP_PASSWORD, and GITHUB_TOKEN to OTA_ENGINE_GITHUB_TOKEN

1.3.0 - 2024-05-22

Full changeset and discussions: #1076.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Enable a service to be renamed into several services when exporting a dataset

1.2.2 - 2024-05-22

Full changeset and discussions: #1075.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Exclude non-record files (e.g., README, LICENSE) from dataset

1.2.1 - 2024-05-09

Full changeset and discussions: #1070.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix ota lint command

1.2.0 - 2024-05-08

Full changeset and discussions: #1069.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add label empty content to reported issues on GitHub when server returns empty content or when PDF content is unextractable
  • Add label invalid selector to reported issues on GitHub when CSS selector is invalid

Changed

  • Consider extraction errors as expected operational errors instead of crashing the engine

Fixed

  • Do not crash when source documents links point to invalid URLs
  • Take into account the --types CLI option
  • Fix counts of terms when they are specified via the CLI option --types
  • Fix display issues for service ID and terms type associated with errors

1.1.3 - 2024-05-02

Full changeset and discussions: #1068.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix validation of modified declarations when declaration history is modified; bug introduced in v1.1.2

1.1.2 - 2024-04-08

Full changeset and discussions: #1066.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix validation of modified declarations when new services were added
  • Fix validation of modified declarations when filters were modified

1.1.1 - 2024-03-28

Full changeset and discussions: #1065.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Ensure to reopen associated issue if terms become inaccessible again

1.1.0 - 2024-03-18

Full changeset and discussions: #1063.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Introduce pluginError error type to isolate plugin errors from the engine and increase robustness

1.0.0 - 2024-03-14

Full changeset and discussions: #1061.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

No code changes were made in this release, it simply signifies that the public API can now be considered stable.

0.37.1 - 2024-02-20

Full changeset and discussions: #1056.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix support of service names with special character in the validation CLI

0.37.0 - 2024-02-20

Full changeset and discussions: #1054.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Minimize required privileges associated to the GitHub token to create issues in the declarations repository

0.36.1 - 2024-02-15

Full changeset and discussions: #1053.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Log a warning in case log emails cannot be sent because of a missing config

0.36.0 - 2024-02-15

Full changeset and discussions: #1052.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Base the loading of the Notifier module on the presence of required configuration, not on the environment; only define the SENDINBLUE_API_KEY environment variable if you want the Notifier to be loaded, avoid relying on NODE_ENV=production

0.35.0 - 2024-02-14

Full changeset and discussions: #1051.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Consider EAI_AGAIN errors as a legitimate part of the tracking lifecycle rather than causing the engine to crash

0.34.3 - 2024-02-13

Full changeset and discussions: #1050.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix engine hang with non-existent service ID

0.34.2 - 2024-02-12

Full changeset and discussions: #1045.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix support of special character in service names

0.34.1 - 2023-12-19

Full changeset and discussions: #1037.

Development of this release was made on a volunteer basis by @Ndpnt.

Fixed

  • Improve performance for large-scale MongoDB databases

0.34.0 - 2023-12-11

Full changeset and discussions: #1033.

Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Expose versions data through the collection API (#1003). When using Git as storage for versions, this API relies on the assumption that the commit date matches the author date, introduced in the engine in June 2022 (#875). If your collection was created before this date, inconsistencies in the API results may arise. You can verify if your version history includes commits with commit dates differing from author dates by executing the following command at the root of your versions repository: git log --format="%H %ad %cI" --date=iso-strict | awk '{if ($2 != $3) print "Author date", $2, "and commit date", $3, "mismatch for commit", $1 }'. You can correct the history with the command: git rebase --committer-date-is-author-date $(git rev-list --max-parents=0 HEAD). Since the entire history will be rewritten, a force push may be required for distributed repositories

Changed

  • Provide a succinct JSON-formatted error message as response in API errors

0.33.1 - 2023-11-28

Full changeset and discussions: #1032.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

0.33.0 - 2023-11-27

Full changeset and discussions: #1031.

Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add terms attribute to /services API response, containing declared term types for each service

0.32.1 - 2023-10-18

Full changeset and discussions: #1026.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Ensure Reporter configuration is defined before instantiating it
  • Ensure to add links to versions and snapshots in issues only if repositories are defined in configuration

Removed

  • No longer define default configuration for Reporter module

0.32.0 - 2023-10-18

Full changeset and discussions: #1025.

Development of this release was supported by GitHub Social Impact through its DPG Open Source Community Manager Program.

Added

  • Auto create labels for declarations repository on GitHub
  • Assign a label based on the error type to issues managed by the engine

Changed

  • Breaking: Revise the formatting of reported issues, with notable adjustments to the title, transitioning from Fix <service_id> - <terms_type> to <service_id> ‧ <terms_type> ‧ not tracked anymore
  • Breaking: Rename tracker module into reporter; update your configuration file by renaming tracker into reporter
  • Breaking: Change configuration for reporter module; update your reporter configuration accordingly

Removed

  • Breaking: No longer assign the label bot-report to issues managed by the engine

0.31.1 - 2023-10-06

Full changeset and discussions: #1024.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Upgrade dependencies patches and ensure compatibility with Node v20

0.31.0 - 2023-09-08

Full changeset and discussions: #1021.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

0.30.1 - 2023-09-06

Full changeset and discussions: #1020.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix validation for removed terms types

0.30.0 - 2023-07-10

Full changeset and discussions: #1015.

Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Embed Swagger UI for graphical user interface documentation of the API; access it on /docs

Changed

  • Breaking: Change path from /services/:serviceId to /service/:serviceId for direct access to a service resource; update paths accordingly in your own codebase
  • Breaking: Change path from /specs to /docs for OpenAPI specification; update paths accordingly in your own codebase and set the HTTP header Accept: application/json

0.29.1 - 2023-06-26

Full changeset and discussions: #1013.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Fix issue creation on GitHub tracker (#1012)

0.29.0 - 2023-06-05

Full changeset and discussions: #1011.

Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.

Added

  • Add collection metadata API (#1003)
  • Add ota serve command to CLI to start the server and expose the API

0.28.0 - 2023-04-25

Changed

  • Include collection name in dataset name

0.27.1 - 2023-04-19

Changed

  • Clean up README

0.27.0 - 2023-04-19

Full changeset and discussions: #996, #999, #1000.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Rename CLI option --terms-types to --types in API; simply rename accordingly in your own codebase
  • Breaking: Rename CLI option --refilter-only, -r to --extract-only, -e in API; simply rename accordingly in your own codebase
  • Breaking: Rename class PageDeclaration to SourceDocument and its atribute noiseSelectors to insignificantContentSelectors in API; simply rename accordingly in your own codebase
  • Breaking: Rename function and its parameters filter({ content, mimeType, pageDeclaration }) to extract(sourceDocument) in API; content and mimeType are embedded sourceDocument attributes; rename accordingly in your own codebase and set content and mimeType in the sourceDocument passed as a parameter to the function
  • Breaking: Rephrase commit messages in Git storage: Start tracking is changed to First record of, Refilter to Apply technical or declaration upgrade on and Update to Record new changes of; existing data will still be loaded, but new commits will use these new messages, if you have scripts that parse commit messages directly, update them accordingly
  • Breaking: Rename document attribute isRefilter to isExtractOnly in MongoDB storage; existing data will still be loaded, but new entries will use this new attribute, if you have scripts that query the Mongo database directly, update them accordingly
  • Make vocabulary consistent throughout the codebase (#971)

Removed

  • Breaking: Remove npm run extract command; use npm run start -- --extract-only instead

0.26.1 - 2023-04-19

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Change URL to engine repo from ambanum/OpenTermsArchive to OpenTermsArchive/engine
  • Replace documentation with links to the documentation website to avoid redundancy

0.26.0 - 2023-02-20

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Split validation and linting of declaration files in 2 commands. ota validate does not test the linting of files anymore.
  • Breaking: ota lint does not fix files by default. ota lint --fix must be used for that.

0.25.2 - 2023-02-20

Full changeset and discussions: #981.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Specification of changelog format

Changed

  • Specification of changelog format is now in CONTRIBUTING

0.25.1 - 2023-02-20

Changed

  • Improve documentation for CLI commands

0.25.0 - 2023-02-08

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Replace behaviour of ota track --schedule. It now tracks the changes only on scheduled hours. It was before running also on launch.

0.24.0 - 2023-01-25

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Trigger a release event on CI to deploy documentation website

0.23.0 - 2023-01-18

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Removed

  • Breaking: Remove obsolete ansible deployment recipes as it was extracted in a dedicated repository. Look at the README to know how to deploy the engine.

0.22.0 - 2023-01-17

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Replace embedded terms types list with the one defined in the dedicated repository @opentermsarchive/terms-types.

0.21.0 - 2023-01-16

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Add dataset command to CLI; this command can be discovered in the documentation and by running ota dataset help

0.20.0 - 2022-12-13

Full changeset and discussions: #959.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Improved reliability and expanded coverage of email protection global filter

0.19.1 - 2022-12-13

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Add missing configuration for production

0.19.0 - 2022-12-13

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Changed

  • Breaking: Unify all CLI commands as subcommands of one single ota command and rename some options; the new CLI can be discovered in the documentation and by running ota help (#978)

0.18.2 - 2022-12-12

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Fixed

  • Ensure paths for given Git storage configuration are relative to current working directory instead of engine module folder

0.18.1 - 2022-12-12

Full changeset and discussions: #979.

Fixed

  • Add missing configuration for production

0.18.0 - 2022-12-12

Full changeset and discussions: #976.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Load both the configurations defined in this module and the configurations defined in the module which use it as dependency

0.17.2 - 2022-12-12

Full changeset and discussions: #977.

Fixed

  • Add missing file required for CLI in the packaging process

0.17.1 - 2022-12-06

Full changeset and discussions: #971.

Fixed

  • Rewrite documentation to bring it up to date with current behaviour

0.17.0 - 2022-12-06

Full changeset and discussions: #957.

Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.

Added

  • Publish package on NPM under name @opentermsarchive/engine
  • Export filter, pageDeclaration and fetch in NPM module
  • Add changelog