Web Worker for background JSON parsing in Langfuse codebase.

In this article, we review how Langfuse parses JSON using web workers in the background. You will learn:

What is Web Worker?
json-parser.worker.ts
useParsedTrace.ts

What is Web Worker?

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down.

A worker is an object created using a constructor (e.g., Worker()) that runs a named JavaScript file — this file contains the code that will run in the worker thread.

Worker

The Worker interface of the Web Workers API represents a background task that can be created via script, which can send messages back to its creator.

Creating a worker is done by calling the Worker("path/to/worker/script") constructor.

Workers may themselves spawn new workers, as long as those workers are hosted at the same origin as the parent page.

Learn more about Web Worker.

json-parser.worker.ts

In the json-parser.worker.ts in Langfuse codebase, you will find this comment.


/**
 * Web Worker for background JSON parsing
*
* Parses large JSON data off the main thread using the iterative
* deepParseJsonIterative function to prevent UI blocking.
*/
import { deepParseJsonIterative } from "@langfuse/shared";
export interface ParseRequest {
id: string;
input: unknown;
output: unknown;
 metadata: unknown;
}
export interface ParseResponse {
  id: string;
  parsedInput: unknown;
  parsedOutput: unknown;
  parsedMetadata: unknown;
  parseTime: number;
}
self.onmessage = function (e: MessageEvent<ParseRequest>) {
  const { id, input, output, metadata } = e.data;
 
  const startTime = performance.now();
try {
    // Parse with high limits since we're off the main thread
    const parsedInput = deepParseJsonIterative(input, {
      maxDepth: Infinity,
      maxSize: 10_000_000, // 10MB
    });
const parsedOutput = deepParseJsonIterative(output, {
  maxDepth: Infinity,
  maxSize: 10_000_000,
});
 
const parsedMetadata = deepParseJsonIterative(metadata, {
  maxDepth: Infinity,
  maxSize: 10_000_000,
});
 const elapsed = performance.now() - startTime;
 
    const response: ParseResponse = {
      id,
      parsedInput,
      parsedOutput,
      parsedMetadata,
      parseTime: elapsed,
    };
  self.postMessage(response);
  } catch (error) {
    console.error("[json-parser.worker] Parse error:", error);
    // Send back unparsed data on error
    self.postMessage({
      id,
 parsedInput: input,
      parsedOutput: output,
      parsedMetadata: metadata,
      parseTime: performance.now() - startTime,
      error: error instanceof Error ? error.message : String(error),
    });
  }
};

Langfuse uses the web workers to parse large JSON. This deepParseJsonIterative function is imported as shown below:


import { deepParseJsonIterative } from "@langfuse/shared";

useParsedTrace.ts

When I searched for json-parse.worker. I found two instances:

In the useParsedTrace.ts, the following code is defined to create a worker


function getOrCreateWorker(): Worker | null {
  if (typeof window === "undefined" || !window.Worker) {
    return null; // SSR or no Worker support
  }
 
  if (!workerInstance) {
    try {
      // Next.js will bundle this as a separate chunk
      workerInstance = new Worker(
        new URL("@/src/workers/json-parser.worker.ts", import.meta.url),
      );
   workerInstance.onmessage = (e: MessageEvent<ParseResponse>) => {
        const callback = pendingCallbacks.get(e.data.id);
        if (callback) {
          callback(e.data);
          pendingCallbacks.delete(e.data.id);
        }
      };
 workerInstance.onerror = (error) => {
        console.error("[useParsedTrace] Worker error:", error);
      };
    } catch (error) {
      console.error("[useParsedTrace] Failed to create worker:", error);
      return null;
    }
  }

You will find that this is used in components/sessions/index.tsx

About me:

Hey, my name is Ramu Narasinga. Email: ramu.narasinga@gmail.com

Tired of AI slop?

I spent 3+ years studying OSS codebases and wrote 350+ articles on what makes them production-grade. I built an open source tool that reviews your PR against your existing codebase patterns.

Your codebase. Your patterns. Enforced.

Get started for free — thinkthroo.com