CI Flaky Timing Luck Agent Skill

Steps that passed by timing luck

Two failures with the same shape, both from a release pipeline:

• A loop wrapped one disk image per architecture. Each iteration mounted a volume under the **same

name**. Releasing a volume is asynchronous, so the next mount either failed as busy or — worse —

quietly mounted under a suffixed name while the script kept writing to the hardcoded path.

• A build step downloaded a native archive through the standard library's URL stream. The download

answered with a redirect to a mirror; on a cross-protocol redirect the stream API saved the

error page body as the target file and returned normally.

Neither is a race you introduced. Both are steps that were always wrong and were being covered

by how fast the machine happened to be. A runner image update changes the timing, and they start

failing — which is why the commit that "breaks" them is usually unrelated.

**The expensive shape is not "fails intermittently", it is "succeeds while producing the wrong

artifact."** A hard failure costs one re-run. A step that reports success and hands the next job a

zero-byte file, an error page, or an empty mount point costs an afternoon, and the traceback points

somewhere it did not come from.

Traps

Never hardcode a mount point — capture the one the tool reports. The tool tells you where it

actually mounted; a name collision changes that answer and nothing else warns you:

# adapted — condensed from the wrapping script, names genericised
MOUNT_POINT="$(hdiutil attach -nobrowse -noautoopen "$RW_DMG" 2>/dev/null \
  | grep -Eo '/Volumes/.*$' | tail -1 || true)"
[[ -d "$MOUNT_POINT" ]] || { echo "Could not mount $RW_DMG" >&2; exit 1; }
VOL_NAME="$(basename "$MOUNT_POINT")"

Everything downstream then derives from $MOUNT_POINT and $VOL_NAME. A script that writes to

/Volumes/<Name> after a silent remount to /Volumes/<Name> 1 writes into the *previous*

iteration's volume and produces an output missing whatever it thought it added.

Clean up before you start, not only after you finish. The previous iteration's cleanup may not

have completed, and a cleanup you run only at the end never runs at all if the body failed:

# adapted — condensed from the wrapping script, names genericised
detach_stale_volumes() {
  for v in /Volumes/<Name> /Volumes/<Name>\ *; do
    [[ -e "$v" ]] && hdiutil detach -force "$v" >/dev/null 2>&1 || true
  done
}
detach_stale_volumes
for attempt in 1 2 3 4 5; do
  MOUNT_POINT="$(hdiutil attach … | grep -Eo '/Volumes/.*$' | tail -1 || true)"
  [[ -d "$MOUNT_POINT" ]] && break
  detach_stale_volumes; sleep 3
done

Note the glob covers the suffixed names too — those are exactly the leftovers a previous collision

created.

Force and retry the release as well. The next iteration starts from whatever this one left

behind, so a best-effort detach at the end of the body is worth a short retry loop.

Prefer a downloader that fails loudly on HTTP errors. The stdlib stream API does not follow a

redirect that switches protocol — it hands back the redirect response itself, and copying that

stream saves the redirect's own HTML page to disk under the artifact's filename. The download looks

like it worked; the checksum is the only thing left to catch it. An external downloader with

explicit flags fails instead:

// adapted
ProcessBuilder(
    "curl", "-fsSL",          // --fail: non-zero on HTTP errors instead of saving the error body
                              // -L: follows redirects across protocols and mirrors
    "--retry", "5", "--retry-delay", "5",
    "--retry-all-errors",     // plain --retry does NOT cover a mid-transfer receive failure;
                              // it retries only HTTP 5xx/408/429 and connection errors
    "-o", target.absolutePath, url,
).inheritIO().start().waitFor()

Check the result even after a zero exit, and delete the partial file. Otherwise an empty or

truncated download is cached and every retry reproduces it:

check(exit == 0 && target.exists() && target.length() > 0) {
    if (target.exists()) target.delete()
    "download failed (exit $exit) for $url"
}

Mirrors are a lottery, so verify content rather than trusting the source. A release URL can

redirect to a different mirror on every request. Pair the download with a digest check against a

value pinned in your source — that is the only assertion that survives a mirror serving something

else.

A cached artifact is a cached failure. Any caching layer in front of a download will happily

preserve a bad file forever. Make the failure path delete, and make the cache key include whatever

identifies the *publication*, not just the version.

Verifying it

Timing-luck steps have tells you can grep for. In shell steps and build scripts, look for:

grep -rn 'Volumes/\|/mnt/\|/tmp/[a-z-]*/' --include='*.sh' --include='*.yml' scripts .github
grep -rn 'openStream\|URL(\|readBytes()' --include='*.kts' --include='*.gradle' .
grep -rn 'sleep ' --include='*.sh' --include='*.yml' scripts .github

• a fixed path where a tool reports its own,
• a stdlib fetch where an HTTP error is possible,
• a bare sleep standing in for a condition — it is a timing assumption with a comment missing.

Then ask of each: if this step did the wrong thing, would the job still be green? Every yes is

a place to add an assertion on the artifact — its existence, its size, its type, its digest — right

where it is produced, rather than letting a later job discover it.