Background jobs for AI apps: queues, retries and idempotency

When should an AI request become a background job?

A short classification may fit inside an HTTP request. A large document import, batch evaluation or multi-step research task can exceed runtime limits or a user's patience. Background processing lets the application accept work, return a status reference and continue independently of the browser connection.

The queue also gives you a place to control concurrency when model-provider limits or database capacity are constrained. It does not make unlimited work affordable. Define what happens when the queue is full, a tenant submits too many jobs or a provider is unavailable.

Use a durable job record and a small message

Persist the job's owner, input reference, state, attempt count and timestamps. Put a job identifier on the queue rather than a large document or a credential. Workers load authorized input from storage and update the job through explicit transitions. The UI can poll or subscribe to status without keeping the original request open.

Consider the boundary between writing the job record and publishing the message. If one succeeds and the other fails, work can be stranded or duplicated. An outbox pattern or a reconciliation process can make that gap recoverable. Define the behavior before relying on the queue as proof that every accepted job will run.

Expect duplicate execution

Standard Amazon SQS queues document at-least-once delivery, including the possibility of receiving a message again. Other queue products have their own guarantees and limits. Read the chosen service's semantics and design the worker so duplicate execution does not duplicate the business action.

Use a stable key for an operation such as generating one invoice or processing one document version. Protect the final side effect with an atomic uniqueness constraint or an external provider's supported idempotency mechanism. Recording that a job started is insufficient if two workers can both pass the check before either commits.

Reference: source 1

Retry transient failures within a budget

Retry errors that may resolve, such as a temporary rate limit, with bounded backoff and jitter. Treat malformed input and missing authorization as terminal until corrected. A dead-letter destination can preserve exhausted jobs for inspection, but it still needs monitoring and a documented reprocessing process.

Repeated model calls can incur additional cost even when your own database writes are idempotent. Set maximum attempts, execution duration and per-job usage. A cancellation request should stop new steps and clearly report whether an external action already occurred. It cannot always undo work that finished elsewhere.

Monitor completion, age and failure reasons

Queue length alone can hide stalled work. Track the oldest waiting job, time to completion, success rate, repeated attempts and dead-letter volume. Alert on customer impact, such as accepted work exceeding its expected completion window, with a named owner and a runbook.

Test a worker crash after output is written but before the message is acknowledged. Test a provider timeout with an uncertain outcome. Those two cases reveal whether the system can recover without duplicate customer actions. Latifold is designed to give coding agents context on supported cloud resources and a governed change path. It does not implement your application's job semantics, idempotency or product-level acceptance checks. Keep those guarantees in application code and test them independently.