Blob Storage

Updated Nov 16, 2026 · 4 min read
On this page

    Students upload homework videos to your course site. The first version stores each video in a database column. Within a month the database is 3 TB, backups take all night, a single 2 GB upload ties up an application server for minutes, and the database's memory is full of video bytes instead of the rows it needs.

    Databases are built for small, structured records that change. Videos, images, PDFs and backups are large, opaque, and written once. They belong in a different kind of store: blob storage (also called object storage), such as Amazon S3, Google Cloud Storage or Azure Blob Storage.

    How it works

    A blob store is a giant key-value store for files. You PUT an object under a key like uploads/u_42/v_981.mp4, and later GET it by that key. There are no folders (the slashes are part of the key), no partial updates (you replace the whole object), and no queries beyond listing keys by prefix.

    Behind the API, the provider splits and copies each object across machines and data centers, so durability is very high (providers advertise many nines) and capacity grows without you thinking about disks.

    Upload: the bytes skip your servers

    Presigned upload: the client asks the Upload Service for an upload, the service records a pending row and returns a presigned PUT URL, the client sends the file directly to the blob store, and an uplo
    Presigned upload: the client asks the Upload Service for an upload, the service records a pending row and returns a presigned PUT URL, the client sends the file directly to the blob store, and an upload-complete event marks the row ready.
    1. The client calls POST /uploads with the file name and size, through the API Gateway and load balancer.
    2. The Upload Service checks the user and quota, creates a metadata row with status pending, and asks the blob store for a presigned URL: a URL that allows one PUT to one key for, say, 15 minutes.
    3. The client uploads the file directly to the blob store with that URL. Your servers never carry the 2 GB.
    4. The blob store emits an "object created" event (or the client calls POST /uploads/{id}/complete). The service marks the row ready and may start processing (thumbnails, transcoding).

    For large files use multipart upload: split the file into parts (say 10 MB each), upload them in parallel, retry only failed parts, then complete. That makes a flaky phone connection survivable.

    Download: metadata from the service, bytes from the CDN

    Download via CDN: the client asks the File Service for file 42, the service checks access in the metadata DB and returns a short-lived signed URL, and the client fetches the bytes from a CDN edge that
    Download via CDN: the client asks the File Service for file 42, the service checks access in the metadata DB and returns a short-lived signed URL, and the client fetches the bytes from a CDN edge that only goes to the blob store on a miss.

    The bucket stays private. The File Service checks permission and hands back a signed URL that expires in minutes. Popular files are served by a CDN in front of the bucket.

    Trade-offs and costs

    PropertyBlob storeDatabase
    Object sizeBytes to terabytesBest under a few KB per row
    Cost per GB-monthLow, lower still in cold tiersMuch higher (fast disks, replicas, memory)
    LatencyTens of ms to first byteAbout 1 ms for an indexed row
    UpdatesReplace whole objectUpdate any column in place
    QueriesGet by key, list by prefixAny indexed query, joins, transactions
    • Two systems to keep in sync: the metadata row and the object can disagree (an upload that never finished, a delete that removed the row but not the object). Run a cleanup job that removes old pending uploads and orphaned objects.
    • Egress cost: reading data out to the internet is billed per GB. A CDN reduces it and speeds it up.
    • Storage classes: cheaper "infrequent" and "archive" tiers cost less to store but more (and slower) to read. Lifecycle rules move objects automatically, for example to archive after 90 days.
    • Security: a public bucket is a classic data leak. Keep buckets private and hand out signed URLs.

    When to reach for it

    • Any user-generated media: photos, videos, audio, documents.
    • Large outputs of jobs: transcoded video, ML models, data exports, logs and backups.
    • Static website assets that a CDN will serve.
    • Anything bigger than about 1 MB that you read and write as a whole.

    Keep in the database: everything you search, filter or update field by field, including the metadata about the blobs.

    Example

    Homework video uploads. Assumptions: 10,000 uploads a day, average 300 MB, kept for 2 years; videos are watched mostly in the first two weeks.

    • Storage: 10,000 × 300 MB = 3 TB per day, about 2.2 PB over 2 years. Only a blob store makes this affordable.
    • Keys: homework/{course_id}/{student_id}/{upload_id}.mp4. Random-looking upload_ids spread load across the store's internal partitions.
    • Upload: multipart, 16 MB parts, 4 in parallel, presigned per part. A dropped connection resumes from the last finished part.
    • Metadata row: uploads(upload_id, student_id, course_id, key, size, status, created_at), so "list my uploads" is a database query, never a bucket listing.
    • Lifecycle: standard tier for 30 days, then infrequent access, then archive after 1 year. Most of the 2.2 PB sits in cheap tiers.
    • Download: teachers get a signed URL through the CDN, valid for 1 hour.

    Common mistakes

    • Storing files in the database. It bloats backups, memory and replication.
    • Proxying uploads through application servers. Use presigned URLs so servers handle only small control requests.
    • Listing the bucket to answer user queries. Keep a metadata table instead.
    • Public buckets or long-lived signed URLs for private content.
    • No cleanup for abandoned uploads and orphaned objects.
    • Forgetting the CDN and egress costs for files many people download.

    Say this in the interview

    "Video bytes go to blob storage and the metadata, owner, key, size and status, goes to Postgres. The client gets a presigned multipart upload URL and sends the file straight to the bucket, so our servers never carry it. When the upload completes, an event marks it ready and kicks off transcoding. Downloads go through the CDN with signed URLs that expire in an hour, and lifecycle rules move videos to colder tiers after 30 days."

    Used in