structure, APIs, occasion loop, concurrency

This weblog submit provides an summary of how Node.js works:

• What its structure seems to be like.
• How its APIs are structured.
  • A couple of highlights of its international variables and built-in modules.
• The way it runs JavaScript in a single thread through an occasion loop.
• Choices for concurrent JavaScript on this platform.

The Node.js platform  

The next diagram supplies an summary of how Node.js is structured:

The APIs accessible to a Node.js app encompass:

• The ECMAScript customary library (which is a part of the language)
• Node.js APIs (which aren’t a part of the language correct):
  • Among the APIs are supplied through international variables:
    • Particularly cross-platform net APIs akin to fetch and CompressionStream fall into this class.
    • However a number of Node.js-only APIs are international, too – for instance, course of.
  • The remaining Node.js APIs are supplied through built-in modules – for instance, 'node:path' (capabilities and constants for dealing with file system paths) and 'node:fs' (performance associated to the file system).

The Node.js APIs are partially applied in JavaScript, partially in C++. The latter is required to interface with the working system.

Node.js runs JavaScript through an embedded V8 JavaScript engine (the identical engine utilized by Google’s Chrome browser).

International Node.js variables  

These are a number of highlights of Node’s international variables:

• crypto provides us entry to a web-compatible crypto API.

• console has a lot overlap with the identical international variable in browsers (console.log() and so on.).

• fetch() lets us use the Fetch browser API.

• course of incorporates an occasion of class Course of and offers us entry to command line arguments, customary enter, customary out, and extra.

• structuredClone() is a browser-compatible operate for cloning objects.

• URL is a browser-compatible class for dealing with URLs.

Extra international variables are talked about all through this weblog submit.

The built-in Node.js modules  

Most of Node’s APIs are supplied through modules. These are a number of continuously used ones (in alphabetical order):

Module 'node:module' incorporates operate builtinModules() which returns an Array with the specifiers of all built-in modules:

import * as assert from 'node:assert/strict';
import {builtinModules} from 'node:module';

const modules = builtinModules.filter(m => !m.startsWith('_'));
modules.kind();
assert.deepEqual(
  modules.slice(0, 5),
  [
    'assert',
    'assert/strict',
    'async_hooks',
    'buffer',
    'child_process',
  ]
);

The totally different kinds of Node.js capabilities  

On this part, we use the next import:

import * as fs from 'node:fs';

Node’s capabilities are available in three totally different kinds. Let’s have a look at the built-in module 'node:fs' for instance:

• A synchronous model with regular capabilities – for instance:
• Two asynchronous kinds:
  • An asynchronous model with callback-based capabilities – for instance:
  • An asynchronous model with Promise-based capabilities – for instance:

The three examples we have now simply seen, show the naming conference for capabilities with comparable performance:

• A callback-based operate has a base title: fs.readFile()
• Its Promise-based model has the identical title, however in a unique module: fsPromises.readFile()
• The title of its synchronous model is the bottom title plus the suffix “Sync”: fs.readFileSync()

Let’s take a more in-depth have a look at how these three kinds work.

Synchronous capabilities  

Synchronous capabilities are easiest – they instantly return values and throw errors as exceptions:

attempt {
  const consequence = fs.readFileSync('/and so on/passwd', {encoding: 'utf-8'});
  console.log(consequence);
} catch (err) {
  console.error(err);
}

Promise-based capabilities  

Promise-based capabilities return Guarantees which are fulfilled with outcomes and rejected with errors:

import * as fsPromises from 'node:fs/guarantees'; 

attempt {
  const consequence = await fsPromises.readFile(
    '/and so on/passwd', {encoding: 'utf-8'});
  console.log(consequence);
} catch (err) {
  console.error(err);
}

Be aware the module specifier in line A: The Promise-based API is positioned in a unique module.

Guarantees are defined in additional element in “JavaScript for impatient programmers”.

Callback-based capabilities  

Callback-based capabilities move outcomes and errors to callbacks that are their final parameters:

fs.readFile('/and so on/passwd', {encoding: 'utf-8'},
  (err, consequence) => {
    if (err) {
      console.error(err);
      return;
    }
    console.log(consequence);
  }
);

This model is defined in additional element in the Node.js documentation.

The Node.js occasion loop  

By default, Node.js executes all JavaScript in a single thread, the foremost thread. The primary thread constantly runs the occasion loop – a loop that executes chunks of JavaScript. Every chunk is a callback and will be thought-about a cooperatively scheduled activity. The primary activity incorporates the code (coming from a module or customary enter) that we begin Node.js with. Different duties are normally added later, resulting from:

• Code manually including duties
• I/O (enter or output) with the file system, with community sockets, and so on.
• And so on.

A primary approximation of the occasion loop seems to be like this:

That’s, the primary thread runs code just like:

whereas (true) { 
  const activity = taskQueue.dequeue(); 
  activity();
}

The occasion loop takes callbacks out of a activity queue and executes them in the primary thread. Dequeuing blocks (pauses the primary thread) if the duty queue is empty.

We’ll discover two subjects later:

• How one can exit from the occasion loop.
• How one can get across the limitation of JavaScript working in a single thread.

Why is that this loop known as occasion loop? Many duties are added in response to occasions, e.g. ones despatched by the working system when enter information is able to be processed.

How are callbacks added to the duty queue? These are frequent potentialities:

• JavaScript code can add duties to the queue in order that they’re executed later.
• When an occasion emitter (a supply of occasions) fires an occasion, the invocations of the occasion listeners are added to the duty queue.
• Callback-based asynchronous operations within the Node.js API observe this sample:
  • We ask for one thing and provides Node.js a callback operate with which it might probably report the consequence to us.
  • Finally, the operation runs both in the primary thread or in an exterior thread (extra on that later).
  • When it’s performed, an invocation of the callback is added to the duty queue.

The next code exhibits an asynchronous callback-based operation in motion. It reads a textual content file from the file system:

import * as fs from 'node:fs';

operate handleResult(err, consequence) {
  if (err) {
    console.error(err);
    return;
  }
  console.log(consequence); 
}
fs.readFile('reminder.txt', 'utf-8',
  handleResult
);
console.log('AFTER'); 

That is the ouput:

AFTER
Don’t neglect!

fs.readFile() executes the code that reads the file in one other thread. On this case, the code succeeds and provides this callback to the duty queue:

() => handleResult(null, 'Don’t neglect!')

Operating to completion makes code easier  

An vital rule for a way Node.js runs JavaScript code is: Every activity finishes (“runs to completion”) earlier than different duties run. We will see that within the earlier instance: 'AFTER' in line B is logged earlier than the result’s logged in line A as a result of the preliminary activity finishes earlier than the duty with the invocation of handleResult() runs.

Operating to completion signifies that activity lifetimes don’t overlap and we don’t have to fret about shared information being modified within the background. That simplifies Node.js code. The following instance demonstrates that. It implements a easy HTTP server:

import * as http from 'node:http';

let requestCount = 1;
const server = http.createServer(
  (_req, res) => { 
    res.writeHead(200);
    res.finish('That is request quantity ' + requestCount); 
    requestCount++; 
  }
);
server.pay attention(8080);

We run this code through node server.mjs. After that, the code begins and waits for HTTP requests. We will ship them through the use of an internet browser to go to http://localhost:8080. Every time we reload that HTTP useful resource, Node.js invokes the callback that begins in line A. It serves a message with the present worth of variable requestCount (line B) and increments it (line C).

Every invocation of the callback is a brand new activity and variable requestCount is shared between duties. Attributable to working to completion, it’s straightforward to learn and replace. There is no such thing as a have to synchronize with different concurrently working duties as a result of there aren’t any.

Why does Node.js code run in a single thread?  

Why does Node.js code run in a single thread (with an occasion loop) by default? That has two advantages:

• As we have now already seen, sharing information between duties is easier if there may be solely a single thread.

• In conventional multi-threaded code, an operation that takes longer to finish blocks the present thread till the operation is completed. Examples of such operations are studying a file or processing HTTP requests. Performing many of those operations is pricey as a result of we have now to create a brand new thread every time. With an occasion loop, the per-operation value is decrease, particularly if every operation doesn’t do a lot. That’s why event-loop-based net servers can deal with larger masses than thread-based ones.

Provided that a few of Node’s asynchronous operations run in threads apart from the primary thread (extra on that quickly) and report again to JavaScript through the duty queue, Node.js will not be actually single-threaded. As a substitute, we use a single thread to coordinate operations that run concurrently and asynchronously (in the primary thread).

This concludes our first have a look at the occasion loop. Be at liberty to skip the rest of this part if a superficial clarification is sufficient for you. Learn on to be taught extra particulars.

The true occasion loop has a number of phases  

The true occasion loop has a number of activity queues from which it reads in a number of phases (you’ll be able to try a few of the JavaScript code within the GitHub repository nodejs/node). The next diagram exhibits crucial ones of these phases:

What do the occasion loop phases do which are proven within the diagram?

• Section “timers” invokes timed duties that have been added to its queue by:

• Section “ballot” retrieves and processes I/O occasions and runs I/O-related duties from its queue.

• Section “examine” (the “instant section”) executes duties scheduled through:

  • setImmediate(activity) runs the callback activity as quickly as attainable (“instantly” after section “ballot”).

Every section runs till its queue is empty or till a most variety of duties was processed. Aside from “ballot”, every section waits till its subsequent flip earlier than it processes duties that have been added throughout its run.

Section “ballot”  

• If the ballot queue will not be empty, the ballot section will undergo it and run its duties.
• As soon as the ballot queue is empty:
  • If there are setImmediate() duties, processing advances to the “examine” section.
  • If there are timer duties which are prepared, processing advances to the “timers” section.
  • In any other case, this section blocks the entire foremost thread and waits till new duties are added to the ballot queue (or till this section ends, see under). These are processed instantly.

If this section takes longer than a system-dependent time restrict, it ends and the following section runs.

Subsequent-tick duties and microtasks  

After every invoked activity, a “sub-loop” runs that consists of two phases:

The sub-phases deal with:

• Subsequent-tick duties, as enqueued through course of.nextTick().
• Microtasks, as enqueued through queueMicrotask(), Promise reactions, and so on.

Subsequent-tick duties are Node.js-specific, Microtasks are a cross-platform net customary (see MDN’s assist desk).

This sub-loop runs till each queues are empty. Duties added throughout its run, are processed instantly – the sub-loop doesn’t wait till its subsequent flip.

Evaluating alternative ways of straight scheduling duties  

We will use the next capabilities and strategies so as to add callbacks to one of many activity queues:

• Timed duties (section “timers”)
  • setTimeout() (net customary)
  • setInterval() (net customary)
• Untimed duties (section “examine”)
  • setImmediate() (Node.js-specific)
• Duties that run instantly after the present activity:
  • course of.nextTick() (Node.js-specific)
  • queueMicrotask(): (net customary)

It’s vital to notice that when timing a activity through a delay, we’re specifying the earliest attainable time that the duty will run. Node.js can’t all the time run them at precisely the scheduled time as a result of it might probably solely examine between duties if any timed duties are due. Due to this fact, a long-running activity could cause timed duties to be late.

Subsequent-tick duties and microtasks vs. regular duties  

Take into account the next code:

operate enqueueTasks() {
  Promise.resolve().then(() => console.log('Promise response 1'));
  queueMicrotask(() => console.log('queueMicrotask 1'));
  course of.nextTick(() => console.log('nextTick 1'));
  setImmediate(() => console.log('setImmediate 1')); 
  setTimeout(() => console.log('setTimeout 1'), 0);
  
  Promise.resolve().then(() => console.log('Promise response 2'));
  queueMicrotask(() => console.log('queueMicrotask 2'));
  course of.nextTick(() => console.log('nextTick 2'));
  setImmediate(() => console.log('setImmediate 2')); 
  setTimeout(() => console.log('setTimeout 2'), 0);
}

setImmediate(enqueueTasks);

We use setImmediate() to keep away from a pecularity of ESM modules: They’re executed in microtasks, which signifies that if we enqueue microtasks on the high degree of an ESM module, they run earlier than next-tick duties. As we’ll see subsequent, that’s totally different in most different contexts.

That is the output of the earlier code:

nextTick 1
nextTick 2
Promise response 1
queueMicrotask 1
Promise response 2
queueMicrotask 2
setTimeout 1
setTimeout 2
setImmediate 1
setImmediate 2

Observations:

• All next-tick duties are executed instantly after enqueueTasks().

• They’re adopted by all microtasks, together with Promise reactions.

• Section “timers” comes after the instant section. That’s when the timed duties are executed.

• We’ve added instant duties through the instant (“examine”) section (line A and line B). They present up final within the output, which signifies that they weren’t executed through the present section, however through the subsequent instant section.

Enqueuing next-tick duties and microtasks throughout their phases  

The following code examines what occurs if we enqueue a next-tick activity through the next-tick section and a microtask through the microtask section:

setImmediate(() => {
  setImmediate(() => console.log('setImmediate 1'));
  setTimeout(() => console.log('setTimeout 1'), 0);

  course of.nextTick(() => {
    console.log('nextTick 1');
    course of.nextTick(() => console.log('nextTick 2'));
  });

  queueMicrotask(() => {
    console.log('queueMicrotask 1');
    queueMicrotask(() => console.log('queueMicrotask 2'));
    course of.nextTick(() => console.log('nextTick 3'));
  });
});

That is the output:

nextTick 1
nextTick 2
queueMicrotask 1
queueMicrotask 2
nextTick 3
setTimeout 1
setImmediate 1

Observations:

• Subsequent-tick duties are executed first.

• “nextTick 2” in enqueued through the next-tick section and instantly executed. Execution solely continues as soon as the next-tick queue is empty.

• The identical is true for microtasks.

• We enqueue “nextTick 3” through the microtask section and execution loops again to the next-tick section. These subphases are repeated till each their queues are empty. Solely then does execution transfer on to the following international phases: First the “timers” section (“setTimeout 1”). Then the instant section (“setImmediate 1”).

Ravenous out occasion loop phases  

The next code explores which sorts of duties can starve out occasion loop phases (stop them from working through infinite recursion):

import * as fs from 'node:fs/guarantees';

operate timers() { 
  setTimeout(() => timers(), 0);
}
operate instant() { 
  setImmediate(() => instant());
}

operate nextTick() { 
  course of.nextTick(() => nextTick());
}

operate microtasks() { 
  queueMicrotask(() => microtasks());
}

timers();
console.log('AFTER'); 
console.log(await fs.readFile('./file.txt', 'utf-8'));

The “timers” section and the instant section don’t execute duties which are enqueued throughout their phases. That’s why timers() and instant() don’t starve out fs.readFile() which experiences again through the “ballot” section (there may be additionally a Promise response, however let’s ignore that right here).

Attributable to how next-tick duties and microtasks are scheduled, each nextTick() and microtasks() stop the output within the final line.

When does a Node.js app exit?  

On the finish of every iteration of the occasion loop, Node.js checks if it’s time to exit. It retains a reference rely of pending timeouts (for timed duties):

• Scheduling a timed activity through setImmediate(), setInterval(), or setTimeout() will increase the reference rely.
• Operating a timed activity decreases the reference rely.

If the reference rely is zero on the finish of an occasion loop iteration, Node.js exits.

We will see that within the following instance:

operate timeout(ms) {
  return new Promise(
    (resolve, _reject) => {
      setTimeout(resolve, ms); 
    }
  );
}
await timeout(3_000);

Node.js waits till the Promise returned by timeout() is fulfilled. Why? As a result of the duty we schedule in line A retains the occasion loop alive.

In distinction, creating Guarantees doesn’t improve the reference rely:

operate foreverPending() {
  return new Promise(
    (_resolve, _reject) => {}
  );
}
await foreverPending(); 

On this case, execution briefly leaves this (foremost) activity throughout await in line A. On the finish of the occasion loop, the reference rely is zero and Node.js exits. Nevertheless, the exit will not be profitable. That’s, the exit code will not be 0, it’s 13 (“Unfinished High-Degree Await”).

We will manually management whether or not a timeout retains the occasion loop alive: By default, duties scheduled through setImmediate(), setInterval(), and setTimeout() maintain the occasion loop alive so long as they’re pending. These capabilities return cases of class Timeout whose methodology .unref() modifications that default in order that the timeout being lively gained’t stop Node.js from exiting. Technique .ref() restores the default.

Tim Perry mentions a use case for .unref(): His library used setInterval() to repeatedly run a background activity. That activity prevented functions from exiting. He mounted the problem through .unref().

libuv: the cross-platform library that handles asynchronous I/O (and extra) for Node.js  

libuv is a library written in C that helps many platforms (Home windows, macOS, Linux, and so on.). Node.js makes use of it to deal with I/O and extra.

How libuv handles asynchronous I/O  

Community I/O is asynchronous and doesn’t block the present thread. Such I/O contains:

• TCP
• UDP
• Terminal I/O
• Pipes (Unix area sockets, Home windows named pipes, and so on.)

To deal with asynchronous I/O, libuv makes use of native kernel APIs and subscribes to I/O occasions (epoll on Linux; kqueue on BSD Unix incl. macOS; occasion ports on SunOS; IOCP on Home windows). It then will get notifications once they happen. All of those actions, together with the I/O itself, occur on the primary thread.

How libuv handles blocking I/O  

Some native I/O APIs are blocking (not asynchronous) – for instance, file I/O and a few DNS providers. libuv invokes these APIs from threads in a thread pool (the so-called “employee pool”). That permits the primary thread to make use of these APIs asynchronously.

libuv performance past I/O  

libuv helps Node.js with extra than simply with I/O. Different performance contains:

• Operating duties within the thread pool
• Sign dealing with
• Excessive decision clock
• Threading and synchronization primitives

As an apart, libuv has its personal occasion loop whose supply code you’ll be able to try within the GitHub repository libuv/libuv (operate uv_run()).

Escaping the primary thread with consumer code  

If we wish to maintain Node.js conscious of I/O, we must always keep away from performing long-running computations in main-thread duties. There are two choices for doing so:

• Partitioning: We will cut up up the computation into smaller items and run every bit through setImmediate(). That permits the occasion loop to carry out I/O between the items.

  • An upside is that we will carry out I/O in every bit.
  • A draw back is that we nonetheless decelerate the occasion loop.

• Offloading: We will carry out our computation in a unique thread or course of.

  • Downsides are that we will’t carry out I/O from threads apart from the primary thread and that speaking with exterior code turns into extra sophisticated.
  • Upsides are that we don’t decelerate the occasion loop, that we will make higher use of a number of processor cores, and that errors in different threads don’t have an effect on the primary thread.

The following subsections cowl a number of choices for offloading.

Employee threads  

Employee Threads implement the cross-platform Internet Employees API with a number of variations – e.g.:

• Employee Threads must be imported from a module, Internet Employees are accessed through a world variable.

• Inside a employee, listening to messages and posting messages is finished through strategies of the worldwide object in browsers. On Node.js, we import parentPort as an alternative.

• We will use most Node.js APIs from employees. In browsers, our selection is extra restricted (we will’t use the DOM, and so on.).

• On Node.js, extra objects are transferable (all objects whose lessons lengthen the inner class JSTransferable) than in browsers.

On one hand, Employee Threads actually are threads: They’re extra light-weight than processes and run in the identical course of as the primary thread.

Then again:

• Every employee runs its personal occasion loop.
• Every employee has its personal JavaScript engine occasion and its personal Node.js occasion – together with separate international variables.
  • (Particularly, every employee is an V8 isolate that has its personal JavaScript heap however shares its working system heap with different threads.)
• Sharing information between threads is proscribed:
  • We will share binary information/numbers through SharedArrayBuffers.
  • Atomics presents atomic operations and synchronization primitives that assist when utilizing SharedArrayBuffers.
  • The Channel Messaging API lets us ship information (“messages”) over two-way channels. The information is both cloned (copied) or transferred (moved). The latter is extra environment friendly and solely supported by a number of information buildings.

For extra info, see the Node.js documentation on employee threads.

Clusters  

Cluster is a Node.js-specific API. It lets us run clusters of Node.js processes that we will use to distribute workloads. The processes are absolutely remoted however share server ports. They’ll talk by passing JSON information over channels.

If we don’t want course of isolation, we will use Employee Threads that are extra light-weight.

Baby processes  

Baby course of is one other Node.js-specific API. It lets us spawn new processes that run native instructions (typically through native shells). This API is roofed in the weblog submit “Executing shell instructions from Node.js”.

Sources of this weblog submit  

Node.js occasion loop:

Movies on the occasion loop (which refresh a few of the background data wanted for this weblog submit):

• “Node’s Occasion Loop From the Inside Out” (by Sam Roberts) explains why working methods added assist for asynchronous I/O; which operations are asynchronous and which aren’t (and must run within the thread pool); and so on.
• “The Node.js Occasion Loop: Not So Single Threaded” (by Bryan Hughes) incorporates a quick historical past of multitasking (cooperative multitasking, preemptive multitasking, symmteric multi-threading, asynchronous multitasking); processes vs. threads; working I/O synchronously vs. within the thread pool; and so on.

libuv:

JavaScript concurrency:

Acknowledgement  

• I’m a lot obliged to Dominic Elm for reviewing this weblog submit and offering vital suggestions.