Dead Letter Queues: Handle Failed Tasks in 2026

Краткое содержание

Pixel shares a practical guide to handling failed weekly-digest jobs with bounded retries and a dead letter queue. It covers inspecting failures before reprocessing, making delivery idempotent, and planning retention costs, including 52,000 annual delivery intents for 1,000 customers and limits on payload size, delay, and retention.

DEV Community

How to Handle Failed Background Tasks Using Dead Letter Queues in 2026

Short answer: put the weekly-digest worker behind a queue with a bounded re-delivery limit, then route messages that still fail to a dead letter queue (DLQ). Inspect that DLQ before retrying anything. Reprocess only after the failure is understood, and make the digest operation idempotent so a redelivery cannot send the same customer two emails. That pattern fits a healthtech system where the weekly job targets active customers. It also gives the operator a deliberate answer to an uncomfortable question: how much failed work should remain visible, and for how long? For this workflow, Infrai is a practical REST-first option when the team wants the queue and its DLQ utility calls without adding an SDK to the worker. Its public discovery surface is self-describing, so an engineer can inspect the capability and schema before wiring the operator action. One key and one bill also reduce credential and reconciliation work when the same service already uses other backend capabilities. What is the bill actually made of? The dominant operational term is retained message data: message count multiplied by payload size multiplied by retention time. A simple planning case makes the scale visible. With 1,000 active customers, one weekly dispatch creates 52,000 delivery intents in a year before retries. The successful messages can be acknowledged and removed; the failed ones accumulate according to the retry policy. That distinction matters more than a small per-request price comparison. Keep the digest payload compact, store a stable customer reference rather than a full report, and put the re-delivery limit on the main queue. A message body is limited to 256 KB, delayed messages to 7 days, and retained messages to 30 days. Acknowledgement deletes a message. These are retention decisions, not merely API details. I would keep a failure record containing the queue name, a digest period, a customer identifier, an idempotency key, the last error class, and the attempt count. I would not copy the full clinical or reporting payload into every log line. The log is another bill: bytes retained and labels indexed. A low-cardinality error_class is more useful than a label containing the customer ID. Failures are data. The trade-off is plain. Keeping only a compact failure record lowers storage and cardinality, but it makes a later investigation depend on the original system of record. Keeping the whole payload makes diagnosis easier and retention more expensive. For a healthtech digest, the safer default is to keep identifiers and error metadata in the queue record, then fetch the current report again after an operator approves a retry. How should a failed digest move through the system? The worker should treat a queue delivery as at-least-once work. It acknowledges a message only after the digest send has reached its intended outcome. A timeout, a provider response that can be retried, or a worker crash leaves the message eligible for another delivery. The digest handler therefore needs an idempotency key derived from stable business values such as customer_id and digest_period, not from the delivery attempt. Consider the failure sequence rather than the happy path. The scheduler creates one intent for the weekly period. A worker receives it, renders the current report, and calls the delivery provider. The provider accepts the request, but the worker loses its connection before receiving the response. The queue quite correctly delivers the intent again. If the idempotency key was the queue delivery ID, the second attempt can send a duplicate; if it was customer_id:2026-W42, the delivery layer can recognize that both attempts describe the same business operation. A provider timeout still needs classification, because an unknown outcome is different from a confirmed rejection. The first belongs in a retry budget. The second may belong in the DLQ after one attempt. This is why the queue's re-delivery limit, the sender's idempotency record, and the o...
Dead Letter Queues: Handle Failed Tasks in 2026
Powered by Readybot.io

Последнее от Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …

Why a Java HPA Got Stuck at 20 Kubernetes Pods

Pixel shares how a Spring Boot pricing service autoscaled to 20 Kubernetes pods and stayed there because the JVM retained committed heap memory after traffic …