A Database, a Cache, and a Queue Remember Different Things

From the team behind aidrop.it — one workspace to build, host, and keep changing your code.

Use a database for records your application treats as authoritative, a cache for copies you can reconstruct, and a queue for work awaiting processing. The choice depends on what losing a record would mean. A missed cache entry can trigger another database read. Losing an accepted order or an unfinished job needs a recovery path.

One purchase illustrates all three roles. You commit the order to PostgreSQL, cache a summary for the account page, and enqueue a receipt job. Each place holds information about the same purchase. You give each a different responsibility, and you handle failure according to that responsibility.

Keep the accepted order in the database

Commit the order and its line items in one database transaction before you confirm acceptance. Later requests can read that record even after you replace the application process. Keep constraints such as unique request identifiers in the database too, so two concurrent requests cannot both create the same logical order through separate application checks.

The order retry contract turns that unique identifier into a check for concurrent submissions, lost responses, and payment retries.

A receipt job should refer to that order by ID. Reconstructing the order from a completed job would tie your business history to queue retention. For a small workload, you can also keep pending jobs in a database table; a separate queue needs a reason.

Rebuild a cache entry from its source

Write a cache entry only when you have another way to obtain the information. On a miss, read the database or recompute the result, then populate the cache again. Decide how much staleness the user can tolerate. Expiry limits the age of a copy; invalidation can remove it when you change the underlying record.

On aidrop.it, the Valkey cache has no persistent volume and uses allkeys-lru. Under memory pressure, Valkey can evict keys according to recent use. That makes it suitable for disposable copies. Keeping the only record of an accepted order there would make its survival depend on cache pressure and restarts.

Keep unfinished work until a worker accounts for it

Treat a queued job as an obligation that a worker must finish or report as failed. Choose a queue library or protocol with acknowledgment, retry, and failure handling. A persistent server cannot supply those application rules by itself. A worker can stop after sending a receipt but before recording that it completed the job.

The aidrop.it queue runs a separate Valkey instance with an append-only file on a persistent volume. Its noeviction policy rejects memory-growing writes at the limit instead of evicting existing jobs. These are distinct Valkey eviction policies, despite the cache and queue using the same engine and protocol.

The queue uses appendfsync everysec. Valkey's persistence documentation describes a potential loss window of about one second of writes after a crash. Persistence therefore needs both a recovery plan and retry-safe workers. It does not promise that a job runs once.

An order commit and a queue write are also separate operations. If your process stops between them, you can save a pending receipt record in the same database transaction as the order, then have a dispatcher retry delivery. Keep a job identifier so repeated delivery does not repeat the business action. The application still owns that coordination.

A storage contract for one order

Write the failure behavior beside each stored representation of a business event. Include what you do when the store refuses a write, as well as what you do when it loses data. Use the record below to review your application's behavior; storage capacity affects whether that behavior remains acceptable under load.

ORDER STORAGE CONTRACT

Database: authoritative order and pending receipt record
  Confirm acceptance after: transaction commits
  Duplicate request: return the existing order
  Recovery: tested database backup

Cache: order summary for the account page
  Missing key: read the database and repopulate
  Stale copy: invalidate on change; choose an expiry
  Cache unavailable: define fallback and protect the database

Queue: receipt job referencing order ID
  Write refused: retain pending work and retry later
  Worker stops: recover unacknowledged work
  Duplicate delivery: use a stable job ID and check completion
  Permanent failure: retain the reason for an operator

FAQ

Can PostgreSQL handle background jobs? Yes. A job table can hold pending work alongside business records. Claiming jobs, retrying failures, and recording completion remain application responsibilities; a separate broker becomes useful when the workload justifies it.

Why does aidrop.it use Valkey for both cache and queue? aidrop.it runs separate instances with different settings. The cache permits eviction and has no persistence; the queue rejects memory-growing writes when full and keeps an append-only log on its own volume.

Does a persistent queue guarantee exactly-once processing? No. A worker may repeat work after a crash or lose confirmation of a completed action. Design acknowledgment, deduplication, and recovery around the effects of each job.