Skip to content
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions .github/workflows/ci-status-watch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: CI status watch

# Posts a Teams message ONLY when one of the README badge workflows on the default
# branch turns red (green -> red). For each it compares the two latest runs with gh; a
# change is reported only if the newest run finished within the look-back window, so a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says the workflow lists the currently red workflows and keeps them listed until they turn green again. This header comment and the code actually implement a green-to-red transition that is deduplicated (a workflow that stays red is reported once, not every day). The comment here is the accurate one, so please update the PR description to match the implementation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be even smarter if we keep posting to our channel every night if something is broken so we get spammed and want to get rid of the spam by fixing the problem and nobody can excuse that he missed the notification ;)

# workflow that stays red is not repeated every day.
Comment thread
hohwille marked this conversation as resolved.
Outdated
# Requires the repository secret TEAMS_WEBHOOK_URL.

on:
workflow_dispatch:
schedule:
- cron: '0 7 * * 1-5' # 07:00 UTC, Mon-Fri (= 09:00 CEST / 08:00 CET)

permissions:
actions: read

jobs:
watch:
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WEBHOOK: ${{ secrets.TEAMS_WEBHOOK_URL }}
REPO: ${{ github.repository }}
BRANCH: ${{ github.ref_name }}
steps:
- name: Detect newly red badge workflows and notify Teams
run: |
set -euo pipefail

# The workflows backing the README badges. Add a line here if a badge is added.
WORKFLOWS="
.github/workflows/build.yml
.github/workflows/update-urls.yml
.github/workflows/nightly-build.yml
.github/workflows/integration-tests.yml
"

# Monday catches up the weekend (Fri-Sun); other weekdays look back 24h.
if [ "$(date -u +%u)" -eq 1 ]; then
SINCE=$(date -u -d '3 days ago' +%Y-%m-%dT%H:%M:%SZ)
else
SINCE=$(date -u -d '1 day ago' +%Y-%m-%dT%H:%M:%SZ)
fi
echo "Checking badge workflows for red since $SINCE"
Comment thread
hohwille marked this conversation as resolved.
Outdated
echo "Resolved branch: $BRANCH"

is_red() { [ "$1" = "failure" ] || [ "$1" = "timed_out" ] || [ "$1" = "startup_failure" ]; }

# Compare each workflow's two latest runs; report it only if it just turned red
# and that run finished inside the window (so a lasting red is reported once).
RED=$(for WF_PATH in $WORKFLOWS; do
ROW=$(gh run list --repo "$REPO" --workflow "$WF_PATH" --branch "$BRANCH" \
--status completed --limit 2 --json conclusion,url,updatedAt,workflowName \
--jq '[(.[0].conclusion // ""), (.[0].updatedAt // ""), (.[0].url // ""), (.[1].conclusion // ""), (.[0].workflowName // "")] | @tsv' || true)
[ -z "$ROW" ] && continue
IFS=$'\t' read -r CUR CURTIME URL PREV NAME <<< "$ROW"
[ -z "$CUR" ] && continue
# lexicographic string comparison sufficient due to ISO-8601 format
[[ "$CURTIME" > "$SINCE" ]] || continue
Comment thread
hohwille marked this conversation as resolved.
Outdated
Comment thread
hohwille marked this conversation as resolved.
Outdated
if is_red "$CUR" && { [ "$PREV" = "success" ] || [ -z "$PREV" ]; }; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A genuinely red run is swallowed when the previous run was cancelled: for runs [failure, cancelled], PREV is cancelled, which is neither success nor empty, so nothing is reported. Since cancelled runs are pruned daily the window is small, but within the same day a real failure can be hidden. Consider treating cancelled as neither red nor green and comparing against the next older non-cancelled run.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@quando632 thanks for this finding that I agree to. 👍
Could you suggest a fix so we can merge?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I was checking this again:

  1. I do not see for which workflow we actually cancel in any normal situation so this would be relevant
  2. If we simply send a notification whenever something is read, we can make this so much simpler and do not actually need to dig in the history at all.

I would therefore suggest to go for KISS and simply send the notification whenever the workflow is failed.

FYI:
Initially I thought we would simply integrate this directly into the according workflow what would be even simpler.
However, the advantage of this dedicated workflow is that this extra nice-to-have feature is isolated, not redundantly copied to multiple workflows, cannot break the critical workflows like nightly-build or especially release so I now like this approach...

jq -n --arg n "$NAME" --arg u "$URL" '{name:$n, url:$u}'
fi
done | jq -s '.')
Comment thread
hohwille marked this conversation as resolved.
Outdated

COUNT=$(echo "$RED" | jq 'length')
if [ "$COUNT" -eq 0 ]; then
echo "No badge workflow turned red. Skipping Teams notification."
exit 0
fi
echo "$COUNT badge workflow(s) turned red."
Comment thread
hohwille marked this conversation as resolved.
Outdated

if [ -z "${WEBHOOK:-}" ]; then
echo "No Teams webhook set, skipping notification."; exit 0
fi

CARD=$(echo "$RED" | jq --argjson count "$COUNT" '{
type: "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
version: "1.4",
body: (
[
{ type: "TextBlock", size: "Large", weight: "Bolder", wrap: true, color: "attention",
text: "🔴 Workflow rot geworden" },
{ type: "TextBlock", spacing: "None", isSubtle: true, wrap: true,
text: (if $count == 1 then "1 Workflow · grün → rot" else "\($count) Workflows · grün → rot" end) }
]
+ [ .[] | {
type: "Container",
separator: true,
spacing: "Medium",
items: [
{ type: "TextBlock", weight: "Bolder", wrap: true, text: "🔴 \(.name)" },
{ type: "TextBlock", spacing: "None", isSubtle: true, size: "Small", wrap: true,
text: "[Run-Log öffnen](\(.url))" }
Comment thread
hohwille marked this conversation as resolved.
Outdated
]
} ]
)
}')

curl -sS --fail -X POST -H "Content-Type: application/json" -d "$CARD" "$WEBHOOK"
154 changes: 110 additions & 44 deletions .github/workflows/issue-pr-observer.yml
Original file line number Diff line number Diff line change
@@ -1,30 +1,55 @@
name: External Issue & PR Observer
name: External Issue, PR & Discussion Observer

# Posts a weekday-morning digest of new EXTERNAL issues/PRs (last 24h) to a Teams
# channel so incoming reports from outside the team are noticed quickly and don't slip.
# "External" = author_association is not MEMBER, OWNER or COLLABORATOR.
# Posts a weekday-morning digest of new EXTERNAL issues, PRs and discussions (last 24h)
# to a Teams channel so incoming activity from outside the team is noticed quickly and
# "External" = author is not in the DAILY_TEAM list below.
# Issues/PRs come from the REST search API, discussions from GraphQL (no REST equivalent).
# Each source falls back to empty on error, so one failing API still posts the rest.
# Requires the repository secret TEAMS_WEBHOOK_URL (a Teams "Workflows" incoming webhook).

on:
workflow_dispatch:
workflow_dispatch:
schedule:
- cron: '0 7 * * 1-5' # 07:00 UTC, Mon-Fri (= 09:00 CEST summer / 08:00 CET winter)

permissions:
issues: read
pull-requests: read
discussions: read
contents: read

jobs:
observe:
runs-on: ubuntu-latest
steps:
- name: Scan for new external issues & PRs and notify Teams
- name: Scan for new external issues, PRs & discussions and notify Teams
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WEBHOOK: ${{ secrets.TEAMS_WEBHOOK_URL }}
REPO: ${{ github.repository }}
# Daily-team logins to exclude; one per line, case-insensitive.
DAILY_TEAM: |
AdemZarrouki
Ali-Shariati-Najafabadi
areinicke
Caylipp
hohwille
JoelAdbu
KarimALotfy
laert-ll
laim2003
marceltchanga9
maybeec
oanding-blrng
Paras14
quando632
shodiBoy1
tineff96
vivu001
run: |
# so a failing gh api trips the || fallback instead of being hidden by the trailing jq
set -o pipefail # no `set -e`: the per-source `|| { ... ITEMS='[]'; }` fallbacks must keep the script running

# Monday catches up the weekend (Fri–Sun); other weekdays look back 24h.
if [ "$(date -u +%u)" -eq 1 ]; then
SINCE=$(date -u -d '3 days ago' +%Y-%m-%dT%H:%M:%SZ)
Expand All @@ -33,55 +58,96 @@ jobs:
fi
echo "Scanning $REPO for items created since $SINCE"

# Search returns both issues and PRs; keep only external authors (not on the team)
# and emit a structured object per item so the card can lay them out nicely.
ITEMS_JSON=$(gh api -X GET search/issues \
-f q="repo:$REPO created:>=$SINCE" \
--jq '[.items[]
| select(.author_association != "MEMBER"
and .author_association != "OWNER"
and .author_association != "COLLABORATOR")
# team logins as a lowercase JSON array
TEAM=$(printf '%s\n' "$DAILY_TEAM" | grep -vE '^[[:space:]]*$' \
| tr -d ' \t\r' | tr '[:upper:]' '[:lower:]' | jq -R . | jq -s .)
echo "Excluding $(echo "$TEAM" | jq 'length') daily-team member(s)."

# Issues + PRs via REST search; drop team authors, keep the rest. On error use [].
ISSUE_ITEMS=$(gh api -X GET search/issues \
-f q="repo:$REPO created:>=$SINCE" -f per_page=100 --jq '.items' \
| jq --argjson team "$TEAM" '[.[]
| select((.user.login | ascii_downcase) as $u | $team | index($u) | not)
| {
kind: (if .pull_request then "Pull Request" else "Issue" end),
icon: (if .pull_request then "🔵" else "🟢" end),
number: .number,
title: .title,
url: .html_url,
user: .user.login
}]')
}]') \
|| { echo "WARN: issue/PR fetch failed, continuing without them."; ISSUE_ITEMS='[]'; }

# No REST search for discussions; take newest 50 via GraphQL, filter by window + team.
OWNER=${REPO%/*}
NAME=${REPO#*/}
DISC_ITEMS=$(gh api graphql \
-f owner="$OWNER" -f name="$NAME" \
-f query='
query($owner:String!, $name:String!) {
repository(owner:$owner, name:$name) {
discussions(first:50, orderBy:{field:CREATED_AT, direction:DESC}) {
nodes { number title url createdAt author { login } }
}
}
}' \
--jq '.data.repository.discussions.nodes' \
| jq --arg since "$SINCE" --argjson team "$TEAM" '[ .[]
| select(.createdAt >= $since)
| select(((.author.login // "") | ascii_downcase) as $u | $team | index($u) | not)
| {
kind: "Discussion",
icon: "💬",
number: .number,
title: .title,
url: .url,
user: (.author.login // "unknown")
} ]') \
|| { echo "WARN: discussion fetch failed, continuing without them."; DISC_ITEMS='[]'; }

COUNT=$(echo "$ITEMS_JSON" | jq 'length')
IP_COUNT=$(echo "$ISSUE_ITEMS" | jq 'length')
D_COUNT=$(echo "$DISC_ITEMS" | jq 'length')
COUNT=$((IP_COUNT + D_COUNT))
if [ "$COUNT" -eq 0 ]; then
echo "No new external issues or PRs. Skipping Teams-notification."
echo "No new external issues, PRs or discussions. Skipping Teams-notification."
exit 0
fi
echo "$COUNT new external issue(s)/PR(s) found."
echo "$COUNT new external item(s): $IP_COUNT issue(s)/PR(s), $D_COUNT discussion(s)."

# Teams' webhook wants a bare Adaptive Card: top-level type must be
# "AdaptiveCard", not wrapped in a message/attachments envelope.
CARD=$(echo "$ITEMS_JSON" | jq --argjson count "$COUNT" '{
type: "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
version: "1.4",
body: (
[
{ type: "TextBlock", size: "Large", weight: "Bolder", wrap: true,
text: "🆕 Neue externe Issues & Pull Requests" },
{ type: "TextBlock", spacing: "None", isSubtle: true, wrap: true,
text: ("Letzter Zeitraum · " + (if $count == 1 then "1 neuer Eintrag" else "\($count) neue Einträge" end)) }
]
+ [ .[] | {
type: "Container",
separator: true,
spacing: "Medium",
items: [
{ type: "TextBlock", weight: "Bolder", wrap: true,
text: "\(.icon) #\(.number) · \(.title)" },
{ type: "TextBlock", spacing: "None", isSubtle: true, size: "Small", wrap: true,
text: "\(.kind) · von @\(.user) · [Öffnen](\(.url))" }
]
} ]
)
}')
# Bare Adaptive Card with one group per source; a group is rendered only if it has
# items, so issues/PRs and discussions stay separated and empty sections are hidden.
CARD=$(jq -n \
--argjson ip "$ISSUE_ITEMS" \
--argjson disc "$DISC_ITEMS" \
--argjson count "$COUNT" '
def section(title; items):
if (items | length) > 0 then
[ { type: "TextBlock", size: "Medium", weight: "Bolder", wrap: true,
spacing: "Large", text: title } ]
+ [ items[] | {
type: "Container", separator: true, spacing: "Medium",
items: [
{ type: "TextBlock", weight: "Bolder", wrap: true,
text: "\(.icon) #\(.number) · \(.title)" },
{ type: "TextBlock", spacing: "None", isSubtle: true, size: "Small", wrap: true,
text: "\(.kind) · von @\(.user) · [Öffnen](\(.url))" }
]
} ]
else [] end;
{
type: "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
version: "1.4",
body: (
[
{ type: "TextBlock", size: "Large", weight: "Bolder", wrap: true,
text: "🆕 Neue externe Aktivität" },
{ type: "TextBlock", spacing: "None", isSubtle: true, wrap: true,
text: ("Letzter Zeitraum · " + (if $count == 1 then "1 neuer Eintrag" else "\($count) neue Einträge" end)) }
]
+ section("📋 Issues & Pull Requests"; $ip)
+ section("💬 Discussions"; $disc)
)
}')

curl -sS --fail -X POST -H "Content-Type: application/json" -d "$CARD" "$WEBHOOK"
Loading