Node.js, Modules, npm and File I/O

Node.js supplies a JavaScript runtime for command-line tools, servers, automation and data processing. Its module systems organize code, npm manages project dependencies and workflows and its file and stream APIs connect programs to operating-system resources.

1. Runtime and dependencies

Node.js runs JavaScript outside the browser. It provides access to server facilities such as files, processes, network sockets and streams; it does not supply the browser DOM as a built-in environment. The REPL evaluates expressions interactively. A module groups related code and controls its public interface. CommonJS uses require and module.exports; ECMAScript modules use import and export. Module format depends on file extensions and package configuration, so the two styles should not be mixed without understanding interoperability.

npm manages packages and scripts. package.json records package metadata, declared dependencies and scripts. A lockfile records resolved dependency information for reproducible installation. Runtime dependencies belong in dependencies; development-only tools commonly belong in devDependencies. Local packages are installed for a project, whereas global tools are installed for broader command-line use. Package names and versions should be deliberate choices rather than copied installation commands.

2. Files and the event loop

File operations have synchronous and asynchronous forms. Synchronous reads block the executing thread. Callback APIs conventionally pass an error as the first callback argument; promise-based APIs integrate with async/await. A stream processes chunks rather than requiring an entire large file in memory. Backpressure prevents a fast producer from overwhelming a slower destination. The event loop coordinates callbacks, but CPU-intensive JavaScript still needs suitable partitioning or worker execution.

import { readFile } from 'node:fs/promises';
const text = await readFile('notes.txt', 'utf8');
console.log(text);

This fragment belongs to an ECMAScript module. Supplying an encoding requests text; without one, file APIs commonly return a Buffer. Paths should be formed with path utilities when portability matters. A missing file is an error to handle, not evidence that an empty file was successfully read.

3. Files, modules and reproducible projects

An error-first callback has the shape (error, result) => { ... }. Check error before using result; the mere fact that the callback ran does not indicate success. Promise rejection serves the corresponding role in promise-based APIs. Write operations can replace existing contents, whereas append operations add to the file, so select the operation according to the intended persistence behavior.

A CommonJS module can assign module.exports = { add } and its caller can obtain add with require. An ECMAScript module uses export function add(...) and an import in its caller. Package scripts provide named workflows such as test and start; a lockfile helps collaborators resolve the same dependency tree. File paths interpreted from the process's working directory differ from paths relative to a module, a common reason code works from one terminal location and fails from another.

4. Runtime, process and command-line input

Node runs JavaScript through the V8 engine and supplies runtime APIs for files, networking, processes, timers, buffers and streams. It does not provide browser globals such as document or a visual page unless a separate library creates an equivalent environment.

The process object exposes information about the current program:

const [, , inputPath, outputPath] = process.argv;

if (!inputPath || !outputPath) {
  console.error('Usage: node report.js <input> <output>');
  process.exitCode = 1;
}

process.argv contains the runtime executable, entry script and supplied arguments. process.env exposes environment variables as strings or undefined. Treat environmental configuration as input: validate required values and do not print secrets in diagnostics.

The REPL—read, evaluate, print, loop—is useful for small experiments. A script file is preferable when behavior must be repeatable, reviewed or tested.

5. CommonJS modules

CommonJS loads modules with require and exposes a public value through module.exports.

// math.cjs
function add(a, b) { return a + b; }
module.exports = { add };

// app.cjs
const { add } = require('./math.cjs');
console.log(add(2, 3));

Each module has its own scope. Required modules are cached after successful loading, so repeated require calls commonly receive the same exported object. This matters when a module stores mutable singleton state.

exports initially references module.exports. Adding exports.add = add works, but assigning exports = { add } only changes the local variable and does not replace the exported value. Use module.exports = ... when replacing it.

6. ECMAScript modules

ES modules use static import and export declarations:

// math.js in a package with "type": "module"
export function add(a, b) { return a + b; }
export const version = '1.0';

// app.js
import { add } from './math.js';

Node determines format through extensions such as .mjs and .cjs, the nearest package type and other resolution rules. Relative ES module specifiers normally include the file extension. Imports are live bindings and static structure enables analysis before evaluation. Dynamic import() returns a promise and can load conditionally.

In an ES module, import.meta.url identifies the module. A path derived from the process working directory solves a different problem from a path located beside the module.

7. Choosing and inter-operating between module systems

Concern	CommonJS	ES modules
Load syntax	require()	import
Export syntax	module.exports	export
Loading style	Historically synchronous	Static graph with asynchronous capabilities
Module location	__filename, __dirname	Derive from import.meta.url
Conditional loading	require() in code	Dynamic import()

Do not mix syntax by trial and error. Confirm package type, file extension, dependency format and the exact import shape. Default and named export mismatches are common interoperability failures.

8. package.json and scripts

{
  "name": "course-report",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "start": "node src/index.js",
    "test": "node --test"
  },
  "dependencies": {
    "some-runtime-package": "^2.4.0"
  },
  "devDependencies": {
    "some-linter": "^9.0.0"
  }
}

The manifest records identity, entry and format configuration, scripts and declared dependency ranges. npm run start executes a named script in the project context. Runtime libraries belong in dependencies; build, formatting, lint and test tools commonly belong in devDependencies.

Packages should usually be installed locally so each project declares and resolves its own version. Global installation is mainly appropriate for deliberately global command-line tools.

9. Versions and reproducible installation

Semantic versions have major, minor and patch components. A caret range commonly allows compatible minor and patch updates within a major version, while the precise effect differs for initial zero versions. A tilde range is narrower. Always interpret the actual declared range rather than assuming “latest.”

package-lock.json records the resolved dependency graph and integrity information. Commit it for applications so collaborators and automated builds can reproduce resolution more reliably. npm ci installs from the lockfile under stricter consistency rules and is useful in clean automation. Reproducibility also depends on runtime versions, platform behavior, native packages and external services.

10. Callback, promise and synchronous file APIs

Node exposes several API styles:

import { readFile as readFileCallback } from 'node:fs';
import { readFile } from 'node:fs/promises';

readFileCallback('notes.txt', 'utf8', (error, text) => {
  if (error) {
    console.error(error);
    return;
  }
  console.log(text);
});

const text = await readFile('notes.txt', 'utf8');

Error-first callbacks put an Error or null-like value first and the result second. Always return or branch after handling the error before using the result. Promise APIs express the same failure as rejection.

Synchronous APIs block the executing thread. They can be reasonable during one-time startup or a short command-line task, but synchronous request-path I/O prevents the server from processing other JavaScript work on that thread.

11. Buffers, text and encoding

Without a text encoding, many file reads return a Buffer, which represents raw bytes. With 'utf8', Node decodes those bytes into a string.

const bytes = await readFile('logo.png');
const text = await readFile('notes.txt', 'utf8');

Do not decode arbitrary binary content as UTF-8. Conversely, byte length and string length are not always the same because one character may occupy several UTF-8 bytes. Specify encodings at system boundaries instead of relying on undocumented assumptions.

12. Writing, appending and atomic intent

writeFile normally replaces a file's contents, while appendFile adds data. Neither name alone guarantees a complete transactional update across crashes or multiple writers.

import { writeFile, rename } from 'node:fs/promises';

await writeFile('report.tmp', reportText, 'utf8');
await rename('report.tmp', 'report.txt');

Writing a temporary file and renaming can reduce the chance of leaving a partially written target on supported filesystems, but robust production handling also considers permissions, same-filesystem rules, concurrent writers, cleanup, durability and platform-specific behavior.

13. Paths and location

Use node:path for platform-aware construction:

import path from 'node:path';
const target = path.join(process.cwd(), 'data', 'report.json');

The working directory comes from how the process was started and can differ from the script directory. In an ES module, convert import.meta.url when a resource must be located relative to the module.

Never concatenate an untrusted filename into a privileged base directory and assume it remains inside that directory. Normalize or resolve, apply an allowlist or generated identifier and verify containment according to the application's threat model. Path traversal segments and absolute paths can escape naive joins.

14. Streams and backpressure

Reading a huge file into one buffer requires memory proportional to the file. Streams process chunks as they arrive.

import { createReadStream, createWriteStream } from 'node:fs';
import { pipeline } from 'node:stream/promises';
import { createGzip } from 'node:zlib';

await pipeline(
  createReadStream('access.log'),
  createGzip(),
  createWriteStream('access.log.gz')
);

Readable streams produce chunks; writable streams consume them; transform streams convert them. Backpressure communicates that the destination cannot accept data as fast as it arrives. pipeline connects streams, propagates failures and handles cleanup more reliably than manually wiring only data events.

Chunk boundaries are not semantic record boundaries. A line or multibyte character can span chunks, so parsers must retain incomplete trailing data between reads or use a suitable transform.

15. Event loop and CPU work

Asynchronous filesystem operations let Node initiate I/O and continue other work until completion is reported. This helps concurrency for waiting operations. It does not make CPU-heavy JavaScript automatically parallel. A large synchronous calculation blocks the event loop and delays every connection handled by that process.

Use efficient algorithms, partition work when suitable, stream data or move genuine CPU workloads to worker threads or separate processes. Measure before adding concurrency; transferring data and coordinating workers also have costs.

16. Error handling and cleanup

Filesystem errors carry codes such as missing path or denied permission. Handle only errors you can interpret meaningfully and propagate the rest with context. Do not convert every error into an empty successful result.

Resource cleanup belongs in structured APIs such as pipeline, finally or explicit close handling. Log enough context to diagnose the operation without revealing secrets. Command-line programs should set a nonzero exit code for failure while allowing buffered diagnostics to complete when possible.

17. Integrated file-report program

import { readFile, writeFile } from 'node:fs/promises';
import path from 'node:path';

async function buildReport(inputName, outputName) {
  const base = path.resolve('data');
  const input = path.resolve(base, inputName);
  const output = path.resolve(base, outputName);
  if (!input.startsWith(base + path.sep) || !output.startsWith(base + path.sep)) {
    throw new Error('Paths must remain inside the data directory');
  }

  const records = JSON.parse(await readFile(input, 'utf8'));
  if (!Array.isArray(records)) throw new TypeError('Expected an array');
  const report = { count: records.length, generatedAt: new Date().toISOString() };
  await writeFile(output, JSON.stringify(report, null, 2) + '\n', 'utf8');
  return report;
}

The program resolves paths, checks containment, decodes text explicitly, separates JSON syntax from shape validation, writes formatted output and propagates errors to its caller.

18. Practical considerations

1. Node does not include the browser DOM by default.
2. CommonJS and ES modules use different export and loading models.
3. Reassigning exports does not replace module.exports.
4. package.json declares ranges; the lockfile records resolutions.
5. A callback running does not imply success; inspect its error first.
6. An encoding returns decoded text; omitting it commonly returns a Buffer.
7. Synchronous I/O blocks the executing thread.
8. Relative paths are often resolved from the process working directory.
9. Streams process chunks and chunks need not align with records.
10. Asynchronous I/O does not make CPU-heavy JavaScript nonblocking.