Configuring Queues

When specifying a queue, you can optionally pass in some options along with the queue object to customize the queue’s behavior.

There’s currently 3 options you can customize:

1. maxAttempts - Default is 1. Increasing this means your job will be retried until currentAttempt <= maxAttempts.
2. payloadTransformer - Transform the payload before it’s inserted into the database and when it’s retrieved from the database.
3. pollingInterval - Default is 2 seconds. This is the interval at which Sidetrack will poll your database for new jobs.

maxAttempts

By increasing the maxAttempts for a job, you are allowing the job to be retried in the future. All jobs have a default maxAttempts of 1, so if you want a job to run at most 5 times before being marked as failed, you can set maxAttempts to 5.

import { Sidetrack } from "sidetrack";
 
const sidetrack = new Sidetrack<{
  userOnboarding: { email: string };
}>({
  databaseOptions: {
    databaseUrl: process.env["DATABASE_URL"]!,
  },
  queues: {
    userOnboarding: {
      run: async (payload, context) => {
        throw new Error("intentionally failing job");
      },
      
maxAttempts?: number | undefinedmaxAttempts: 5,
    },
  },
});
 
sidetrack.start();
 
sidetrack.insertJob("userOnboarding", { email: "a@example.com" });

This job will fail on the first run then move to retrying status since its currentAttempt (1) is less than 5. After 5 attempts, the job will transition to a status of failed.

payloadTransformer

The payloadTransformer option allows you to transform the payload before it’s inserted into the database and when it’s retrieved from the database. It can be set on the queue or the sidetrack instance. When set on the sidetrack instance, the same transformer will be used for all queues. The queue transformer will take precedence over the sidetrack instance transformer.

import { Sidetrack } from "sidetrack";
import superjson from "superjson";
 
const sidetrack = new Sidetrack<{
  userOnboarding: { registeredAt: Date };
}>({
  databaseOptions: {
    databaseUrl: process.env["DATABASE_URL"]!,
  },
  // Transform Date objects when saving to and loading from the database
  payloadTransformer: {
    serialize: (payload) => superjson.stringify(payload),
    deserialize: (payload) => superjson.deserialize(payload),
  },
  queues: {
    userOnboarding: {
      run: async (payload) => {
        // payload.registeredAt is a Date object, not a string
        console.log(`User registered at ${payload.registeredAt.toISOString()}`);
      },
      payloadTransformer: {
        serialize: (payload) => superjson.stringify(payload),
        deserialize: (payload) => superjson.deserialize(payload),
      },
    },
  },
});
 
sidetrack.start();
 
// Date objects are automatically handled by the transformer
sidetrack.insertJob("userOnboarding", { registeredAt: new Date() });

By default, when storing job payloads in the database, they are serialized to JSON. This works fine for simple objects, but can cause issues with special types like Date objects which get converted to strings. Using a payloadTransformer, you can customize how the payload is serialized and deserialized.

In this example, we use superjson  to handle the transformation. This allows us to work with Date objects directly in our job handlers, while ensuring they are properly stored in the database.

pollingInterval

The pollingInterval option allows you to customize the interval at which Sidetrack will poll your database for new jobs. It can be set on the queue or the sidetrack instance. If set on the sidetrack instance, the same polling interval will be used for all queues. The queue polling interval will take precedence over the sidetrack instance polling interval.

import { Sidetrack } from "sidetrack";
 
const sidetrack = new Sidetrack<{
  userOnboarding: { email: string };
}>({
  databaseOptions: {
    databaseUrl: process.env["DATABASE_URL"]!,
  },
  queues: {
    userOnboarding: {
      run: async (payload, context) => {
        throw new Error("intentionally failing job");
      },
      
pollingInterval?: PollingInterval | undefinedpollingInterval: 3000, // 3 seconds
    },
  },
  
SidetrackOptions<{ userOnboarding: { email: string; }; }>.pollingInterval?: PollingInterval | undefinedpollingInterval: 5000, // 5 seconds
});
 
sidetrack.start();
 
sidetrack.insertJob("userOnboarding", { email: "a@example.com" });