Node.js loop through files in directory [SOLVED]

This tutorial guides you through Node.js looping, the fs module, and practical examples of Node.js loop through files in directory.

By the end of the tutorial, you should comfortably loop through files using while, for, and for-await-of loops with the fs module's methods like opendirSync(), readdir(), and fs.promises.opendir().

Let's get started.

 

How to perform a loop in Node.js

Looping in Node.js can be hectic if you fail to understand the relationship between synchronous/asynchronous code with loops.

A loop repeats a block of code as long as a specified condition. The three main types of loops to use in Node.js are while, for, and for-await-of.

 

while loop

The while loop separates the initializer, condition, and increment. Here is an example of a while loop.

let i = 0
while (i < 5) {
  console.log(i)
  i++
}

Where;

• let i = 0: initializer,
• i < 5: condition, and
• i++: increment.

The initializer marks the start of a loop; the condition ends it, so it doesn't run forever, while an increment moves the loop from one variant to the next.

 

for loop

The for loop combines the initializer, condition, and increment in a statement.

for (let i = 0; i < 5; i++) {
  console.log(i)
}

The for loop further abstracts the three variants (initializer, condition, and increment) depending on whether we are looping enumerables or iterables.

Elements of enumerables (objects) are accessible via properties, while elements of iterables (arrays) are accessible via values. A for-in loop loops through enumerables, whereas a for-of loop loops through iterables.

// enumerable
const person = { username: "Doe", age: 23, codes: false }
for (let value in person ) {
  console.log(value)
}
// iterable
const learner = ["Doe", 23, false]
for (let detail of learner ) {
  console.log(detail)
}

 

for await of loop

JavaScript code can run synchronously or asynchronously. A delay in executing a synchronous code portion halts the execution of subsequent portions of the program.

On the other hand, a delay in the execution of asynchronous code does not halt the execution of subsequent code portions. And that is the challenging part about looping asynchronous code in Node.js.

The loop tries to run through elements as fast as possible such that it may not wait for asynchronous code to return data before manipulating the data. So, by the time the asynchronous code starts running, the loop has ended.

That is where the for-await of loop comes in. Unlikely the while and for loops, it waits for asynchronous code to run. For example, we could control code execution with a timer, callback function, or a promise.

// asynchronous code
const delay = (ms) => {
    return new Promise( resolve => {
        setTimeout(() => {
            resolve(`You delayed for ${ms/1000} ${ms > 1000 ? 'seconds' : 'second'}.`)
        }, ms)
    })
}

// an array of asynchronous code
const delays = [delay(3000), delay(1000), delay(5000)]

// loop through the array
const asyncDelay = async () => {
    for await (let d of delays) 
        console.log(d)
}
asyncDelay()

 

Using fs module to loop through files in directory in Node.js

The fs module enables reading or writing to the filesystem.

The built-in module has multiple functions to simplify working with the file system. You can view all the methods and properties by entering the REPL mode and typing fs followed by a dot . and a double-press on the Tab key.

For example,

fs.closeSync(): closes a file descriptor and the associated file.

fs.opendir(): opens the given directory in the file system.

fs.readSync(): reads and returns a file's contents.

fs.readdir(): reads a directory's contents.

The methods ending in Sync are synchronous, while those lacking the Sync suffix asynchronously achieve the same effect as their synchronous counterparts.

Using the fs.promises property, we can also asynchronously interact with the hard disk. However, this time around, we mainly return a promise that could either be resolved or rejected.

For example, fs.promises.opendir() asynchronously opens a directory without nesting multiple callback functions.

 

Lab environment setup

Create the project directory with script files and a subdirectory with 5 files, as follows.

mainDirectory/
├── files
│   ├── file1.js
│   ├── file2.js
│   ├── file3.txt
│   ├── file4.py
│   ├── file5.js
│   └── file.txt
├── opendirSync.js
├── promisesOpendir.js
└── readdir.js

I create the project structure on Ubuntu.

$ mkdir mainDirectory && cd mainDirectory
$ touch opendirSync.js promisesOpendir.js readdir.js
$ mkdir files && cd files
$ touch file.txt file1.js file2.js file3.txt file4.py file5.js

We will implement file looping as follows:

• opendirSync.js: synchronous file (while) looping,
• readdir.js: callback file (for-of) looping, and
• promisesOpendir.js: promise-based file (for-await-of) looping.

In the examples section, you can write the respective content using any code editor and then run the file with the node command.

node [file]

 

Practical examples of Node.js loop through files in directory

Let's loop through the files using the fs module's methods we discussed earlier.

 

Example-1: Using opendirSync + while loop

Input

// in opendirSync.js
const fs = require('fs')

const directory = fs.opendirSync('files')
let file
while ((file = directory.readSync()) !== null) {
  console.log(file.name)
}
directory.closeSync()

We import the fs module. We then synchronously open the files directory. Before closing the directory, we loop through its contents, console-logging each file's name.

Output

$ node opendirSync.js 
file1.js
file4.py
file3.txt
file5.js
file2.js
file.txt

We get the 6 files inside the files directory.

 

Example-2: Using callbacks (readdir + for-of loop)

Input

// in readdir.js
const fs = require("fs")

fs.readdir('files', (err, files) => {
    if (err) throw err
    for (let i of files) console.log(i)
})

process.on("uncaughtException", e => e ? console.log(e.message) : console.log(""))

First, we import the fs module. Next, we asynchronously read the files directory, returning a callback that either throws an error or prints the directory's content.

We loop through the files using the for-of loop, console-logging each file. Lastly, we handle the exception using the process module.

Output

$ node readdir.js 
file.txt
file1.js
file2.js
file3.txt
file4.py
file5.js

The system prints the files in alphabetical order.

 

Example-3: Using promises (opendir + for-await-of loop)

Input

const fs = require('fs')

const readFiles = async (directory) => {
  const files = await fs.promises.opendir(directory)
  for await (const file of files) console.log(file.name)
}

readFiles('files').catch(e => console.log(e))

Using the asynchronous readFiles() function, we open and read a chunk of files periodically using fs.promises.opendir() method and store the resulting object in the files variable.

Using the for-await-of loop, we dissect a file from the object and console-log it. Lastly, we call the custom readFiles() function and handle exceptions with the catch block.

Output

The system randomly prints the files.

$ node promisesOpendir.js 
file1.js
file4.py
file3.txt
file5.js
file2.js
file.txt

The best part about promise-based looping is that it works quickly for a large number of files.

 

Summary

I could find alot of questions posted on stackoverflow.com wherein users wanted to understand how to loop through files in directory in Node.js and JavaScript.

Looping through files in a folder Node.JS
Iterate over files in a folder (javascript)
how can I loop through files in a folder and subfolders using electron?

So I decided to have this tutorial. A Node.js loop through files in directory is possible through the while, for, and for-await-of loop with the fs module's methods like opendirSync(), readdir(), and fs.promises.opendir().