Blob Store Agent Skill

Blob store

Store large, immutable, unstructured objects — images, video, backups, model

weights, document blobs — in a flat namespace keyed by a string, replicated for

durability and served by direct download. Getting it wrong means stuffing

multi-megabyte blobs into a row-oriented database (where they bloat the working

set, wreck cache locality, and cap throughput) or hand-rolling a file server that

loses data on the first disk failure.

When to reach for this

Objects are large (KB to GB), written once and read many times, and you only ever

fetch them whole by key — never query *inside* them. Photo/video stores, user

uploads, backups, data-lake/ML datasets, static-site assets, log archives. The

access pattern is PUT key → GET key, durability matters, and the total volume is

too large or too cold to sit in a primary database.

When NOT to

Small structured records you query, filter, sort, or join — that is data-storage.

Data that needs transactions, secondary indexes, or partial updates (blobs are

replace-whole, not edit-in-place). Low-latency reads of tiny values (a KV cache or

caching wins). A few files on one box that never grow — the local filesystem is

fine; a blob store is operational overhead you do not need yet (YAGNI). Naming

"object storage" for a workload that is really a database is failure mode #2.

Clarify first

• Object size distribution — average and p99 size? (Decides chunking, multipart

thresholds, and whether reads stream or buffer.) → back-of-the-envelope.

• Read:write ratio and access recency — write-once/read-many? How fast does data

go cold? (Drives tiering and CDN fronting.)

• Durability and availability target — how many nines of durability? Can a read

briefly fail or must it always succeed? (Replication vs erasure coding, multi-region.)

• Access control — public, private, or time-limited per-object grants? (Signed URLs.)
• Mutability and history — do objects change? Must old versions be retained

(compliance, undo)? (Versioning + lifecycle.)

• Egress profile — who reads, from where, how often? (CDN offload, egress cost.)

The options

Durability scheme (how many copies, what shape)

• N-way replication — store N full copies on different nodes/racks/AZs. Use when

objects are small, hot, and latency matters; simplest to reason about.

• Erasure coding (EC) — split an object into *k* data + *m* parity shards; any *k*

reconstruct it. Use for large/cold data at scale — same durability as replication

at ~1.4x overhead instead of 3x. (Mechanics in references/deep-dive.md.)

Storage tier (price/latency/retrieval trade)

• Hot/standard — millisecond reads, highest $/GB. Use for actively served objects.
• Cool/infrequent — cheaper storage, retrieval fee/slightly higher latency. Use

for backups and data read a few times a month.

• Archive/cold — cheapest storage, minutes-to-hours retrieval. Use for compliance

retention and rarely-touched data; never for anything on a request path.

Upload path

• Single PUT — one request. Use for small objects (under the multipart threshold).
• Multipart / resumable — split into parts, upload in parallel, retry per-part,

commit on completion. Use for large objects and flaky networks; the default above

the threshold.

Mutation model

• Immutable + versioning — each write is a new version; deletes are tombstones.

Use when history, undo, or accidental-overwrite protection matters.

• Overwrite-in-place (last-writer-wins) — simplest; no history. Use when only the

latest object matters and storage of old copies is waste.

Trade-offs

| Option | What it solves | What it worsens | Change it when |

|---|---|---|---|

| N-way replication | Simple, fast reads, fast rebuild | 3x+ storage cost | Data is large/cold and cost dominates → erasure coding |

| Erasure coding | Same durability at ~1.4x storage | CPU + multi-node read on every fetch; slow small-object reads; costly rebuild | Objects are small/hot and latency matters → replication |

| Hot tier | Low-latency serving | Highest $/GB | Data goes cold and is rarely read → cool/archive |

| Archive tier | Cheapest at-rest storage | Minutes–hours to first byte; retrieval fees | Anything ends up on a latency-sensitive path → hot/cool |

| Multipart/resumable upload | Large files survive flaky links; parallel throughput | More client logic; orphaned parts cost money | Objects are small → single PUT |

| Versioning | Undo, history, overwrite protection | Storage grows silently; needs lifecycle expiry | Only latest matters → overwrite, last-writer-wins |

| Signed URLs | Offload transfer off your app; scoped access | Leaked/over-broad URLs; clock-skew expiry bugs | Content is fully public → CDN + public read |

Behavior under stress

A blob store rarely "falls over" the way a database does, but it amplifies trouble in

specific ways.

• Hot object / hot prefix: a viral file or a key scheme where many writes share a

prefix concentrates load on one partition. *Mitigate:* front hot reads with a CDN

(content-delivery), randomize/hash key prefixes, replicate the hot object.

• Metadata-index bottleneck: the index that maps key → shard locations is the real

SPOF and the throughput ceiling (millions of tiny objects hurt far more than a few

huge ones). *Mitigate:* shard the index, cache hot lookups, prefer fewer-larger objects.

• Thundering herd on cold-tier promotion: a burst of reads for archived data

triggers slow, expensive bulk retrievals. *Mitigate:* expose retrieval as async, queue

it (messaging-streaming), set expectations on latency.

• Orphaned multipart uploads: aborted uploads leave parts that silently accrue cost.

*Mitigate:* lifecycle rule to abort incomplete uploads after N days.

• Egress storm / cost blow-up: a popular object served directly from origin saturates

bandwidth and runs up egress bills. *Mitigate:* CDN in front; the origin should serve

cache fills, not end users.

• Partial failure during write: a node dies mid-write. *Mitigate:* write is not

acknowledged until the durability quorum (replicas or EC shards) is met; background

repair re-replicates under-durable objects.

Monitor: request rate and error rate per operation (PUT/GET/DELETE), p99 first-byte

latency, durability/repair queue depth, per-prefix hotness, incomplete-multipart count,

and egress volume + cost.

How to apply

• Clarify the inputs — pin object size distribution, read:write ratio, durability

target, access control, and egress profile (see *Clarify first*). If the data is

really small structured records you query, stop — use data-storage.

• Pick from the trade-off table — choose a durability scheme (replication for

small/hot, erasure coding for large/cold), a default tier, an upload path keyed to

size, and a mutation model keyed to whether history matters.

• Set the key knobs — design the key/prefix scheme to avoid hotspots, set the

multipart threshold and part size, define lifecycle rules (tier transitions, version

expiry, abort-incomplete-uploads), and decide signed-URL TTLs.

• Stress-test the choice — walk each item in *Behavior under stress* (hot object,

metadata bottleneck, cold-retrieval herd, orphaned parts, egress storm) and confirm a

mitigation exists for the ones this traffic can trigger.

• Size it with numbers — total storage with replication/EC overhead, object count

(metadata-index load), peak PUT/GET QPS, and monthly egress. → *Numbers that matter*.

• Pick a provider — default to the generic recipe; open a provider file only if the

user named a cloud (see *Choosing a provider*).

• Show the trade-off in the answer — give a `Decision | Solves | Worsens |

Change it when…` table with one row per choice you made here (e.g. the

option, its store, its failure policy); the last column is a concrete trigger

(a number or observable signal). Then name what breaks first.

Dos and don'ts

Do

• Keep the blob in the store and a *pointer* (key + metadata) in the database — never the

bytes in a row.

• Treat objects as immutable; version instead of editing in place when history matters.
• Front public/hot reads with a CDN so the origin serves fills, not end users.
• Use multipart/resumable upload above the size threshold, and abort incomplete uploads via lifecycle.
• Design key prefixes to spread load; hash or shuffle high-cardinality prefixes.
• Set lifecycle rules to move cold data down tiers and expire old versions automatically.

Don't

• Don't store large blobs in a relational/NoSQL row — it wrecks the working set and throughput.
• Don't put archive tiers on a read path; minutes-to-hours retrieval will time out requests.
• Don't serve a popular object directly from origin without a CDN (egress blowup).
• Don't hand out broad or long-lived signed URLs; scope them tight and short.
• Don't assume erasure coding is free — it adds CPU, multi-node reads, and expensive rebuilds.
• Don't ignore millions-of-tiny-objects: the metadata index, not the disks, is the ceiling.

Numbers that matter

The figures that drive the design: total stored bytes × durability overhead (replication

≈ 3x, erasure coding ≈ 1.3–1.5x), object *count* (the metadata index scales with count, not

size), peak upload/download QPS, and monthly egress (often the dominant cost). Object-store

durability targets are commonly quoted around eleven nines; treat that as a design goal set

by the durability scheme, not a given. For the actual storage/bandwidth/QPS arithmetic and

unit conversions, use back-of-the-envelope — do not restate its tables here.

Interface sketch

The contract is small and key-addressed:

PUT    /{bucket}/{key}        body=bytes, headers: Content-Type, optional checksum
GET    /{bucket}/{key}        → bytes (supports Range for partial/streamed reads)
DELETE /{bucket}/{key}        → tombstone (new version if versioning on)
HEAD   /{bucket}/{key}        → metadata only (size, etag, version-id)
# multipart: Initiate → UploadPart×N (parallel, retryable) → Complete | Abort
# signed URL: presign(GET|PUT, key, expiry) → time-limited URL the client uses directly

The key is the whole index (e.g. userId/2026/photo-uuid.jpg); choose it for both

access pattern and prefix spread. The etag/checksum lets clients verify integrity and

do conditional requests. The database stores this key plus app metadata, not the bytes.

Choosing a provider

Default to the generic recipe above. If the user names a cloud, read

references/providers/<provider>.md for the managed-service mapping, quotas/limits, and

provider-specific trade-offs. If no file exists for that provider, the generic recipe is

the answer.

Diagram

To visualize the upload/download path (client → signed URL → store; CDN fronting GET; index

mapping key → shards) or the EC write fan-out, use the in-plugin architecture-diagram

skill — show the metadata index as a distinct node and the CDN as the edge layer. Do not

embed Mermaid here.

Related building blocks

• content-delivery — *pairs with* this: a CDN fronts the blob origin so repeat reads are

served from the edge and the store handles only cache fills (edge caching is *owned* there).

• data-storage — *alternative to* this for large unstructured objects: store the blob here,

keep a pointer (key + metadata) in the DB; sharding/indexing the metadata is *owned* there.

• back-of-the-envelope — *depends on* this for storage, object-count, QPS, and egress sizing

(the numbers that pick replication vs EC and a tier live there).

• messaging-streaming — *pairs with* this to queue async work like cold-tier retrieval and

post-upload processing (transcode, thumbnail); delivery guarantees are *owned* there.

• caching — *pairs with* this for hot small-object reads and metadata-lookup offload.
• system-design — *owned-concept lives in* the orchestrator: the reasoning loop, the

trade-off method, and the ten failure modes.

References

• references/deep-dive.md — chunking, the metadata index, erasure-coding math and read

path, durability/repair, versioning + lifecycle, multipart internals, signed-URL mechanics.

Read when designing the store in detail.

• references/providers/{generic,aws,azure,gcp}.md — service mappings, decision-changing

limits, and pitfalls per environment.