BullMQ reference

In a nutshell

BullMQ is a Node.js library that provides a queuing system built on top of Redis.

Key terms

Term	Description
Queue	A collection of jobs waiting to be processed. You may create multiple queues for different types of jobs (e.g., emails, reports, file system clean up) or for jobs with different priorities (e.g., queue for paying customers users vs non-paying users).
Job	A task that needs to be performed (e.g., send an email, delete unused files, generate a report). You add jobs to a queue.
Worker	A Unix process that spawns to perform jobs. Need at least one worker in order to process jobs. You can have multiple workers and BullMQ will distribute jobs across them. You assign workers to process a queue.

Install

npm i bullmq

You also need to install and run Redis to use BullMQ.

Usage

1. Create a queue

import { Queue } from 'bullmq';
const myQueue = new Queue('foo');

2. Add jobs to a queue

async function addJobs() {
  await myQueue.add('myJobName', { foo: 'bar' });
  await myQueue.add('myJobName', { qux: 'baz' });
}
await addJobs();

3. Create worker(s) to process a queue

import { Worker } from 'bullmq';
const worker = new Worker(queueName, async job => {
  // Will print { foo: 'bar'} for the first job
  // and { qux: 'baz' } for the second.
  console.log(job.data);
});

4. Listen for worker events

4.1 Listen for events of a specific worker

worker.on('completed', job => {
  console.log(`${job.id} has completed!`);
});
worker.on('failed', (job, err) => {
  console.log(`${job.id} has failed with ${err.message}`);
});

4.2 Listen for events of all workers

import { QueueEvents } from 'bullmq';
const queueEvents = new QueueEvents();
queueEvents.on('waiting', ({ jobId }) => {
  console.log(`A job with ID ${jobId} is waiting`);
});
queueEvents.on('active', ({ jobId, prev }) => {
  console.log(`Job ${jobId} is now active; previous status was ${prev}`);
});
queueEvents.on('completed', ({ jobId, returnvalue }) => {
  console.log(`${jobId} has completed and returned ${returnvalue}`);
});
queueEvents.on('failed', ({ jobId, failedReason }) => {
  console.log(`${jobId} has failed with reason ${failedReason}`);
});

Features / other notes

• You can retry failed jobs.
• You can assign priorities to queues.
• BullMQ provides automatic recovery from process crashes.

Sources

• BullMQ docs
• taskforcesh/bullmq GitHub repo