Skip to content

handle duplicate notifications sent for the same book from different tasks - #515

Merged
elfkuzco merged 1 commit into
mainfrom
hanlde-duplicate-notifications
Sep 14, 2026
Merged

elfkuzco merged 1 commit into
mainfrom
hanlde-duplicate-notifications

Conversation

@elfkuzco

Copy link
Copy Markdown
Contributor

Changes

  • add task_id column to zimfarm_notifications table
  • require task_id when processing notifications from zimfarm
  • when new notification arrives and comes with a different task ID (which happens when user tries to upload an already processed book), allow it to pass but processing logic now marks it as bad notification instead of ignoring it as a duplicate notification. It is impossible to create a book with that notification ID as that would cause an integrity error (since book's PK is notification ID).
  • mark possible title uploads coming from duplicate notification as failed in CMS (even though they aren't on Zimfarm). This quickly allows it to be processed by background tasks to delete manually upload zim from S3
  • re-design zimfarm notification view with the same style as book view
  • retreive notifications by combination of ZIM ID and task ID instead of just ZIM ID
  • correctly associate title upload with the same book that zimfarm sent since we now have the same task ID

This fixes #494

@elfkuzco
elfkuzco force-pushed the hanlde-duplicate-notifications branch from 6b0d5ea to 48f0a22 Compare September 11, 2026 01:44
@elfkuzco elfkuzco self-assigned this Sep 11, 2026
@codecov

codecov Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.61538% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.75%. Comparing base (784affd) to head (1e786bd).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...ms_backend/mill/processors/zimfarm_notification.py 69.23% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #515      +/-   ##
==========================================
- Coverage   81.78%   81.75%   -0.03%     
==========================================
  Files          69       69              
  Lines        4397     4412      +15     
  Branches      515      518       +3     
==========================================
+ Hits         3596     3607      +11     
- Misses        640      642       +2     
- Partials      161      163       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@elfkuzco
elfkuzco requested a review from benoit74 September 11, 2026 01:50
Comment thread backend/src/cms_backend/db/book.py
Comment thread backend/src/cms_backend/db/book.py
Comment thread backend/src/cms_backend/db/book_location.py
Comment thread backend/src/cms_backend/mill/processors/zimfarm_notification.py Outdated
Comment thread backend/src/cms_backend/mill/processors/zimfarm_notification.py Outdated
Comment thread backend/src/cms_backend/mill/processors/zimfarm_notification.py Outdated
@elfkuzco
elfkuzco requested a review from benoit74 September 14, 2026 07:31

@benoit74 benoit74 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elfkuzco
elfkuzco force-pushed the hanlde-duplicate-notifications branch from 4fe3929 to 1e786bd Compare September 14, 2026 11:51
@elfkuzco
elfkuzco merged commit 4f0abd4 into main Sep 14, 2026
7 checks passed
@elfkuzco
elfkuzco deleted the hanlde-duplicate-notifications branch September 14, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fail ZIM upload when ZIM has already been processed

2 participants