Edge-Native Background Tasks: Building Resilient Workers with Cloudflare Queues

Edge computing (like Cloudflare Workers or Vercel Edge Functions) has completely solved the problem of global, ultra-low latency API delivery. However, running heavy, long-running asynchronous background tasks—like processing image uploads, sending welcome emails, compiling sitemaps, or executing batch database syncing—presents a major serverless challenge.

Edge functions carry strict execution time limits (typically 10 to 30 seconds). If you attempt to run intensive background tasks inside your standard edge HTTP request-response cycle, the function will be abruptly terminated by the host platform, resulting in lost data or failed updates.

To build robust serverless systems, we must decouple our HTTP endpoints from our background tasks using a reliable message broker.

In 2026, the absolute best tool for this is Cloudflare Queues. Built natively over Cloudflare's serverless infrastructure, it enables you to queue up asynchronous messages at the edge with at-least-once delivery guarantees, automatic retries, and high durability—completely eliminating the need to maintain expensive, heavy background servers (like RabbitMQ or Amazon SQS).

In this architectural guide, we'll build a highly resilient, edge-native image-metadata processing queue using Cloudflare Workers and Cloudflare Queues.

⚡ 1. The Producer-Consumer Edge Architecture

Our architecture is split into two lightweight, serverless workers connected via a Cloudflare Queue:

[User Upload] ──> [1. HTTP Producer Worker] ──(Pushes Message)──> [ Cloudflare Queue ]
                                                                       │ (Batches & Pulls)
[Updates Database] <── [2. Consumer Worker] <─────────── (Invokes process() Loop)

1. 2.
   The Producer Worker: A standard edge HTTP worker that receives user upload metadata. It performs quick validations, instantly pushes a transaction payload message to the Queue, and returns a 202 Accepted response to the user in less than 15ms.
2. 4.
   The Cloudflare Queue: Automatically buffers messages, handles concurrency throttle rates, and manages retries if failures occur.
3. 6.
   The Consumer Worker: A background worker invoked automatically by the queue. It pulls batches of messages asynchronously, processes them (e.g. fetching images and updating database records), and acknowledges successful completions.

🏗️ 2. Step 1: Writing the Producer Worker

The producer is a standard ES Modules worker. It utilizes the bound queue object inside its env context to push messages safely:

typescript
export interface Env {
  // Bind our Cloudflare Queue resource
  IMAGE_PROCESSING_QUEUE: Queue<any>;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 });
    }

    try {
      const payload = await request.json() as { imageUrl: string; userId: string };

      // Validate input parameters
      if (!payload.imageUrl || !payload.userId) {
        return new Response("Missing parameters", { status: 400 });
      }

      console.log("Queueing image processing task at the edge...");

      // 1. Push payload message directly to Cloudflare Queue
      await env.IMAGE_PROCESSING_QUEUE.send({
        userId: payload.userId,
        imageUrl: payload.imageUrl,
        timestamp: Date.now(),
      });

      // 2. Return immediate success response to user
      return new Response(JSON.stringify({ status: "Accepted", message: "Task queued successfully!" }), {
        status: 202,
        headers: { "Content-Type": "application/json" },
      });

    } catch (err: any) {
      return new Response(err.message, { status: 500 });
    }
  }
};

🛠️ 3. Step 2: Writing the Consumer Worker

The consumer is a background worker. Instead of a fetch() handler, it exports a queue() handler, which receives batches of messages programmatically:

typescript
export interface MessagePayload {
  userId: string;
  imageUrl: string;
  timestamp: number;
}

export default {
  // Automatically invoked by Cloudflare Queues
  async queue(batch: MessageBatch<MessagePayload>, env: any): Promise<void> {
    console.log(`Retrieved batch containing ${batch.messages.length} messages.`);

    // Loop through each message in the batch
    for (const message of batch.messages) {
      try {
        const { userId, imageUrl, timestamp } = message.body;

        console.log(`[Processing] User: ${userId}, Image: ${imageUrl}, Queued at: ${timestamp}`);

        // 1. Execute heavy background task (e.g. calling an AI vision API, resizing, etc.)
        await processImageMetadata(imageUrl, userId);

        // 2. Acknowledge successful processing. The message is safely deleted from the queue.
        message.ack();

      } catch (error) {
        console.error("Failed to process message:", error);
        
        // If an error is thrown and we don't call ack(), 
        // Cloudflare will automatically retry delivering this message based on retry configs!
        message.retry();
      }
    }
  }
};

async function processImageMetadata(url: string, user: string) {
  // Simulate heavy edge worker processing (e.g. database updates, Webhooks, etc.)
  await new Promise((resolve) => setTimeout(resolve, 800));
  console.log(`Successfully completed processing for user ${user}`);
}

🚀 4. Configuring Wrangler (wrangler.toml)

To deploy this distributed queue architecture, you define the Queue resource binding inside your project's wrangler.toml configuration file:

toml
# /wrangler.toml
name = "edge-image-producer"
main = "src/index.ts"
compatibility_date = "2026-05-31"

# 1. Bind our queue so our producer worker can push to it
[[queues.producers]]
  queue = "image-processing-queue"
  binding = "IMAGE_PROCESSING_QUEUE"

# 2. Bind our consumer worker class to consume messages
[[queues.consumers]]
  queue = "image-processing-queue"
  max_batch_size = 10        # Batch up to 10 messages per invocation
  max_batch_timeout = 5      # Or wait up to 5 seconds before invoking
  max_retries = 3            # Auto-retry failed messages up to 3 times

🏁 5. Conclusion: Resilient Serverless Workflows

By leveraging Cloudflare Workers and Cloudflare Queues, serverless applications can safely execute computationally-heavy, long-running asynchronous tasks without blocking client-facing HTTP response loops. Pushing messages to highly durable, edge-native buffers guarantees that even under massive traffic spikes or backend API downtime, your data remains perfectly safe, automatically retrying until successful execution. The result is a resilient, edge-native microservice architecture that is incredibly cost-efficient and highly reliable.