fix: update Rugby Borough Council scraper

[CJskii]

Summary

Updates the Rugby Borough Council module to use the Cloud9 waste collections API now used by the live Rugby Borough Council bin day page.

Rugby no longer exposes the old Liferay CollectionDayFinderPortlet form that the previous Selenium scraper depended on. The live page now embeds a Cloud9 web component:

<c9-waste-collections client-id="rugby"></c9-waste-collections>

That component calls:

https://apps.cloud9technologies.com/rugby/citizenmobile/webapi/wastecollections/{uprn}

This PR replaces the broken Selenium selector flow with the same API endpoint used by the live page.

Fixes

Fixes #2147

Testing

Tested locally with a public Rugby UPRN example:

python uk_bin_collection/uk_bin_collection/collect_data.py \
  RugbyBoroughCouncil \
  "https://www.rugby.gov.uk/check-your-next-bin-day" \
  -p "CV22 5EA" \
  -u "100070193903" \
  -w "http://localhost:4444/wd/hub"

Output:

{
    "bins": [
        {
            "type": "Blue-lid recycling bin",
            "collectionDate": "10/07/2026"
        },
        {
            "type": "Food caddy",
            "collectionDate": "15/07/2026"
        },
        {
            "type": "Black refuse bin",
            "collectionDate": "17/07/2026"
        }
    ]
}

Also ran:

python -m py_compile uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py

Summary by CodeRabbit

• New Features

  • Waste collection dates for Rugby Borough Council are now retrieved directly from a live data source for faster, more reliable updates.
  • Bin collection entries are displayed in chronological order for easier planning.

• Bug Fixes

  • Improved formatting and handling of collection dates to ensure consistent, correct schedules.
  • Reduced reliance on browser-based page loading, helping prevent scraping-related failures.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5cc3126d-686d-4beb-854c-478c2296d58d

📥 Commits

Reviewing files that changed from the base of the PR and between 36dc60c and cba3349.

📒 Files selected for processing (1)

• uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py

🚧 Files skipped from review as they are similar to previous changes (1)

• uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py

---

📝 Walkthrough

Walkthrough

RugbyBoroughCouncil.py now fetches bin collection data from a JSON API using uprn, filters container entries, formats collection dates, and sorts the resulting bins chronologically. The Selenium and HTML table scraping flow is removed.

Changes

API-based data retrieval

Layer / File(s)	Summary
Replace scraper with API call uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py	Imports switch to requests, and parse_data now validates uprn, requests the Rugby API, extracts wasteCollectionDates, formats ISO dates, builds data["bins"], and sorts them by collection date.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• robbrad/UKBinCollectionData#1723: Updates a council-specific parse_data path to consume a JSON API and adjust bin/date extraction to the response schema.
• robbrad/UKBinCollectionData#1742: Also replaces Selenium/BeautifulSoup scraping with a direct JSON API call and rebuilds the bins list from the response.

Suggested reviewers: dp247

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: updating the Rugby Borough Council scraper.
Linked Issues check	✅ Passed	The PR replaces the failing Selenium flow with the Rugby API and returns collection dates, matching the issue's fix goal.
Out of Scope Changes check	✅ Passed	The change stays focused on the Rugby Borough Council scraper and its API migration, with no unrelated edits.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py`:
- Around line 18-22: In RugbyBoroughCouncil.py, the
`requests.get(api_url).json()` call should not parse blindly because it can hang
or accept bad responses. Update the fetch in the Rugby Borough Council
collection logic to pass a timeout to `requests.get`, then check the response
status with a failure path before calling `.json()`. If the response is not OK,
raise or otherwise fail explicitly in the same method so unexpected
endpoint/UPRN issues are surfaced early instead of being treated as empty bin
data.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d879da6c-a8d2-4cbf-a76f-d21cf89b2c19

📥 Commits

Reviewing files that changed from the base of the PR and between b65502c and 36dc60c.

📒 Files selected for processing (1)

• uk_bin_collection/uk_bin_collection/councils/RugbyBoroughCouncil.py