CI status watch - #2091
Conversation
Coverage Report for CI Build 30564081540Coverage increased (+0.03%) to 72.618%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions1 previously-covered line in 1 file lost coverage.
Coverage Stats💛 - Coveralls |
…1/IDEasy into feature/ci-status-watch
|
|
||
| # Posts a Teams message ONLY when one of the README badge workflows on the default | ||
| # branch turns red (green -> red). For each it compares the two latest runs with gh; a | ||
| # change is reported only if the newest run finished within the look-back window, so a |
There was a problem hiding this comment.
The PR description says the workflow lists the currently red workflows and keeps them listed until they turn green again. This header comment and the code actually implement a green-to-red transition that is deduplicated (a workflow that stays red is reported once, not every day). The comment here is the accurate one, so please update the PR description to match the implementation.
There was a problem hiding this comment.
Maybe it would be even smarter if we keep posting to our channel every night if something is broken so we get spammed and want to get rid of the spam by fixing the problem and nobody can excuse that he missed the notification ;)
| IFS=$'\t' read -r CUR CURTIME URL PREV NAME <<< "$ROW" | ||
| [ -z "$CUR" ] && continue | ||
| [[ "$CURTIME" > "$SINCE" ]] || continue | ||
| if is_red "$CUR" && { [ "$PREV" = "success" ] || [ -z "$PREV" ]; }; then |
There was a problem hiding this comment.
A genuinely red run is swallowed when the previous run was cancelled: for runs [failure, cancelled], PREV is cancelled, which is neither success nor empty, so nothing is reported. Since cancelled runs are pruned daily the window is small, but within the same day a real failure can be hidden. Consider treating cancelled as neither red nor green and comparing against the next older non-cancelled run.
There was a problem hiding this comment.
@quando632 thanks for this finding that I agree to. 👍
Could you suggest a fix so we can merge?
There was a problem hiding this comment.
Actually I was checking this again:
- I do not see for which workflow we actually cancel in any normal situation so this would be relevant
- If we simply send a notification whenever something is read, we can make this so much simpler and do not actually need to dig in the history at all.
I would therefore suggest to go for KISS and simply send the notification whenever the workflow is failed.
FYI:
Initially I thought we would simply integrate this directly into the according workflow what would be even simpler.
However, the advantage of this dedicated workflow is that this extra nice-to-have feature is isolated, not redundantly copied to multiple workflows, cannot break the critical workflows like nightly-build or especially release so I now like this approach...
Co-authored-by: quando632 <quang-hieu.do@capgemini.com> Co-authored-by: Jörg Hohwiller <hohwille@users.noreply.github.com>
There was a problem hiding this comment.
@hohwille as requested, here is the concrete change for the KISS approach report whatever is currently red, every morning, instead of detecting a green→red transition. Four suggestions below, all in ci-status-watch.yml;
One thing worth deciding explicitly rather than leaving it as-is: you wrote "every night", but the cron is 0 7 * * 1-5. Something breaking on Friday evening stays silent until Monday 09:00 CEST. 0 7 * * * would close that gap weekday-only is defensible too, I just don't think it should stay that way by accident. Not included in the suggestions since it's your call.
Thanks for pointing that out. Indeed I did not want to imply a workflow schedule/cron change as we do not work on weekends and need no notification on Saturday/Sunday. So I was just lax/un-precise and you are fully correct. |
Co-authored-by: quando632 <quang-hieu.do@capgemini.com>
hohwille
left a comment
There was a problem hiding this comment.
@quando632 thanks for your constructive review suggestions following my KISS idea that I all applied 👍
I have one last suggestion, I would consider to apply before merge.
Can you quickly double-check that I am not missing something.
Then we finally merge and take this nice feature live.
| # long as it stays red it is reported every morning, so nobody can miss it. | ||
| RED=$(for WF_PATH in $WORKFLOWS; do | ||
| gh run list --repo "$REPO" --workflow "$WF_PATH" --branch "$BRANCH" \ | ||
| --status completed --limit 5 --json conclusion,url,updatedAt,workflowName \ |
There was a problem hiding this comment.
you increased the limit from 2 to 5. However, we only take the first result [0] and ignore the rest anyway.
IMHO the old limit was more reasonable (could be even 1).
| --status completed --limit 5 --json conclusion,url,updatedAt,workflowName \ | |
| --status completed --limit 2 --json conclusion,url,updatedAt,workflowName \ |
This PR adds ci status watch and external discussion observer workflows
Implemented changes:
ci-status-watch.ymlweekday-morning workflow that lists the workflows currently red onmainand posts them as a single Teams card, so a broken CI is noticed in the Daily. Reports current state (a workflow stays listed until it's green again); creates no issues. Read-only (actions: read).issue-pr-observer.ymlthe daily digest now also reports new external GitHub Discussions (fetched via GraphQL, since they have no REST search). Issues/PRs and discussions are shown as two separate sections in the same card. Each source falls back to empty on error, so a failure in one still posts the other.Testing instructions
TEAMS_WEBHOOK_URLis set (Teams incoming webhook).mainis red, a card with the list appears in Teams; otherwise the run log says all green and nothing is posted.Checklist for this PR
Make sure everything is checked before merging this PR. For further info please also see
our DoD.
mvn clean testlocally all tests pass and build is successful#«issue-id»: «brief summary»(e.g.#921: fixed setup.bat). If no issue ID exists, title only.In Progressand assigned to you or there is no issue (might happen for very small PRs)with
internal