Skip to content
Last updated on February 28, 2023
10 min read

Streaming Data in Edge Functions

Learn how to fetch streamable data in Edge Functions to deliver faster responses using the Streams Web API.

Vercel Edge Functions support the Streams Web API, which allows you to read, write, transform, encode, and decode streams. You can also set your function responses to be streams, which will cause them to respond as data is transmitted in chunks, rather than wait until all the data is fetched.

Edge Functions must begin sending a response within 30 seconds to fall within the maximum initial response time. Once a reply is made, the function can continue to run. This means that you can use Edge Functions to stream data, and the response will be delivered as soon as the first chunk of data is available.

How to stream data in an Edge Function

To create an Edge Function that returns a readable stream, your Function's exported handler method should return a Response object that takes an instance of the ReadableStream interface as the first argument in its constructor. For example:

pages/api/streaming-example.ts
export const config = {
  runtime: 'edge',
};
 
export default async function handler() {
  const encoder = new TextEncoder();
 
  const customReadable = new ReadableStream({
    start(controller) {
      controller.enqueue(encoder.encode('Basic Streaming Test'));
      controller.close();
    },
  });
 
  return new Response(customReadable, {
    headers: { 'Content-Type': 'text/html; charset=utf-8' },
  });
}

The following example demonstrates an Edge Function that returns a ReadableStream object which contains an HTML page. The page says "Vercel Edge Functions + Streaming", and this code works in both JavaScript and TypeScript.

streaming-example.ts
export const config = {
  runtime: 'edge',
};
 
export default async function handler() {
  const encoder = new TextEncoder();
 
  const readable = new ReadableStream({
    start(controller) {
      controller.enqueue(
        encoder.encode(
          '<html><head><title>Vercel Edge Functions + Streaming</title></head><body>',
        ),
      );
      controller.enqueue(encoder.encode('Vercel Edge Functions + Streaming'));
      controller.enqueue(encoder.encode('</body></html>'));
      controller.close();
    },
  });
 
  return new Response(readable, {
    headers: { 'Content-Type': 'text/html; charset=utf-8' },
  });
}

Running asynchronous processes on streamed data

You can use the Streams API to run asynchronous processes on streamed data. For example, you can use the TransformStream interface to transform data asynchronously as it is being streamed (e.g. how ChatGPT Plugins work). The following example demonstrates how to use a TransformStream to convert the output of a ReadableStream using an async function:

transform-stream-example.ts
import {
  createParser,
  ParsedEvent,
  ReconnectInterval,
} from 'eventsource-parser';
 
export const config = {
  runtime: 'edge',
};
 
export default async function handler() {
  const encoder = new TextEncoder();
  const decoder = new TextDecoder();
 
  const res = await fetch('https://example.com/really-big-data-stream');
 
  const readableStream = new ReadableStream({
    async start(controller) {
      function onParse(event: ParsedEvent | ReconnectInterval) {
        if (event.type === 'event') {
          const data = event.data;
          if (data === '[DONE]') {
            // Signal the end of the stream
            controller.enqueue(encoder.encode('[DONE]'));
          }
          // feed the data to the TransformStream for further processing
          controller.enqueue(encoder.encode(data));
        }
      }
 
      const parser = createParser(onParse);
      // https://web.dev/streams/#asynchronous-iteration
      for await (const chunk of res.body as any) {
        parser.feed(decoder.decode(chunk));
      }
    },
  });
 
  const transformStream = new TransformStream({
    async transform(chunk, controller) {
      const content = decoder.decode(chunk);
      if (content === '[DONE]') {
        console.log('done, closing stream...');
        controller.terminate(); // Terminate the TransformStream
        return;
      }
      const results = await someAsyncFunction(content);
      controller.enqueue(encoder.encode(`${content}: ${results}`));
    },
  });
 
  return new Response(readableStream.pipeThrough(transformStream), {
    headers: { 'Content-Type': 'text/html; charset=utf-8' },
  });
}

Caveats

When you are using streams in a busy production system, chances are the chunks you receive on the client side will be fragmented. For example, you'll receive responses like this:

{ "data": { "key" : "valu

And then a moment later, the rest of the response is streamed in:

e" } }

This is because streams get broken up based on the behavior and distance of the physical infrastructure between the communicating parties. The shorter the path between the parties, the less likely chunks get broken up. The longer the path the more likely chunks get broken up.

Because of this, chunks can get broken up into fragments when streaming in production - even when they could stay as a single chunk in localhost.

To solve this, you can use a parser like eventsource-parser or you can also queue in the chunks on the client side and wait for a full JSON object to be received before processing it.

Here's an example:

pages/stream.tsx
/* ... */
 
const response = await fetch('/api/stream', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
});
if (!response.ok) {
  throw new Error('Failed to fetch');
}
 
const data = response.body;
if (!data) {
  throw new Error('No data received');
  return;
}
 
const reader = data.getReader();
const decoder = new TextDecoder();
let done = false;
let tempValue = ''; // temporary value to store incomplete json strings
 
while (!done) {
  const { value, done: doneReading } = await reader.read();
  done = doneReading;
  let chunkValue = decoder.decode(value);
 
  // if there is a temp value, prepend it to the incoming chunk
  if (tempValue) {
    chunkValue = tempValue + chunkValue;
    tempValue = '';
  }
 
  // match json string and extract it from the chunk
  const match = chunkValue.match(/\{(.*?)\}/);
  if (match) {
    tempValue = chunkValue.replace(match[0], '');
    chunkValue = match[0];
  }
 
  try {
    const data = JSON.parse(chunkValue);
    /* do something with the data */
  } catch (e) {
    // store the incomplete json string in the temporary value
    tempValue = chunkValue;
  }
}
 
/* ... */

Constraints

The amount of data you can stream in an Edge Function is limited by its memory allocation, and execution initial response time. Exceeding these limits will cause your function to fail.

Learn more

To learn more about the Streams API, check out some code samples on web.dev: