How to Parse CSV Data in NodeJS [Practical Examples]

Introduction

A Comma Separated Values (CSV) file is a text file whose values are separated by commas. CSV files store tabular data where each of the lines in the file represents a record. The records have values separated by commas that represent fields. One of the most common uses of CSV files is to import and export data. In this article, you will learn how to parse CSV files in NodeJS. By the end, you will be able to read and write CSV files.

 

Prerequisites

To better understand this article and follow along with the example, you should have prior knowledge of JavaScript concepts and at least Node 8 installed.

 

Getting Started

Start by initializing your NodeJs project by generating a package.json file by running the following command.

npm init -y

Create a new file and call it, `csvsample.csv`.This will be the file containing the CSV sample data. We will be using mockaroo to generate some random data. You can choose to generate your own or copy the following.

id,first_name,last_name,email,gender,ip_address
1,Isabeau,Bramwell,ibramwell0@studiopress.com,Female,66.102.183.228
2,Gipsy,Frickey,gfrickey1@redcross.org,Bigender,43.218.105.94
3,Hunt,Rabbage,hrabbage2@hp.com,Male,180.122.171.137
4,Wilden,Wolters,wwolters3@psu.edu,Female,234.223.66.241
5,Kalie,Hayley,khayley4@mozilla.org,Female,66.163.172.74

 

Parsing CSV Data

You can parse a CSV file using different methods. This article covers the following ways:

• Read the entire CSV file as a string.
• Read the CSV file line by line.
• Use CSV parser node module.

 

Method-1: Read the entire CSV file as a string

In this method, you will read the entire file as a string and then split it into rows and columns.

To get started, create a file called readCSV1.js and import the file system module.

const fs = require("fs");

The file system module will help you interact with other files, in this case, the file with the sample data.

The next step is to read the entire file as a string using the readFileSync command which returns the content of the file passed. Passing in the UTF-8 encoding option returns the file content as a string and not a buffer.

let data = fs.readFileSync("./csvsample.csv", "utf8");

You will then split the data into rows which will then be split into columns. Consider the code below:

data = data.split(/\r?\n/);

In this code, we are using a regular expression to split the data string at a new line which essentially creates an array of rows (records). To create columns, you will need a loop that will traverse through the data array and split the comma-separated values into individual columns(fields).

for (let i in data) {
 data[i] = data[i].split(",");
}

 

Method-2: Read the CSV File Line by Line

You may have noticed the shortcoming of reading the entire CSV file at once. Sooner or later, you will run out of memory. This can be avoided by reading the file line by line. The readLine module provides an interface for reading data line by line. In your readCSV2.js file, add the following code to import the fs and  the readline module

const fs = require("fs"),
const readline = require("readline");

To create a readable steam, use `rl.createInterface()` and pass the CSV sample data as the input.

const input = fs.createReadStream("./csvsample.csv");
const rl = readline.createInterface({input});

Now to read the file one line at a time using the following code.

let data = [];
rl.on("line", (row) => {
 data.push(row.split(","));
});
rl.on("close", () => {
 console.log(data);
});

 

Method-3: Use a CSV Parser Node Module

Lastly, you can make the process of parsing CSV files easier by using one of the numerous CSV parser modules. In this post, we will be using the csv-parser module.

To get started, install csv-parser

npm i csv-parser

To parse the file, create readCSV3.js and add the following code

[
 {
   id: '1',
   first_name: 'Isabeau',
   last_name: 'Bramwell',
   email: 'ibramwell0@studiopress.com',
   gender: 'Female',
   ip_address: '66.102.183.228'
 },
 {
   id: '2',
   first_name: 'Gipsy',
   last_name: 'Frickey',
   email: 'gfrickey1@redcross.org',
   gender: 'Bigender',
   ip_address: '43.218.105.94'
 },
 {
   id: '3',
   first_name: 'Hunt',
   last_name: 'Rabbage',
   email: 'hrabbage2@hp.com',
   gender: 'Male',
   ip_address: '180.122.171.137'
 },
 {
   id: '4',
   first_name: 'Wilden',
   last_name: 'Wolters',
   email: 'wwolters3@psu.edu',
   gender: 'Female',
   ip_address: '234.223.66.241'
 },
 {
   id: '5',
   first_name: 'Kalie',
   last_name: 'Hayley',
   email: 'khayley4@mozilla.org',
   gender: 'Female',
   ip_address: '66.163.172.74'
 }
]

In the first two lines, you are importing the fs and csv-parser modules. The fs module lets you interact with the file system and the csv-parser module is the package you will use to read the CSV file.

To start, create a readable stream from the csvsample.csv file using fs.createReadStream() and pass in the path to the CSV file. Follow this by piping the created stream to the csv() function. As each row is read, the data event is emitted and you get access to the row in the callback function. You can use this data to perform other operations like filtering the data. These rows do not include the header row. To get access to the headers, you will have to pass in the `{ headers: true }` argument to the csv() function. In the code above, we are just pushing it to an array. Once all the rows have been parsed, the `end` event is emitted and the data read is logged to the console. You should see the data below if you run the code.

[
 {
   id: '1',
   first_name: 'Isabeau',
   last_name: 'Bramwell',
   email: 'ibramwell0@studiopress.com',
   gender: 'Female',
   ip_address: '66.102.183.228'
 },
 {
   id: '2',
   first_name: 'Gipsy',
   last_name: 'Frickey',
   email: 'gfrickey1@redcross.org',
   gender: 'Bigender',
   ip_address: '43.218.105.94'
 },
 {
   id: '3',
   first_name: 'Hunt',
   last_name: 'Rabbage',
   email: 'hrabbage2@hp.com',
   gender: 'Male',
   ip_address: '180.122.171.137'
 },
 {
   id: '4',
   first_name: 'Wilden',
   last_name: 'Wolters',
   email: 'wwolters3@psu.edu',
   gender: 'Female',
   ip_address: '234.223.66.241'
 },
 {
   id: '5',
   first_name: 'Kalie',
   last_name: 'Hayley',
   email: 'khayley4@mozilla.org',
   gender: 'Female',
   ip_address: '66.163.172.74'
 }
]

 

Analyzing the Parsed Data

What if you only want the full names of the users in the CSV file? Consider the following block of code:

const dataArr = [];
fs.createReadStream("./csvsample.csv")
 .pipe(csv())
 .on("data", (data) => {
   const fullName = {
     firstName: data.first_name,
     lastName: data.last_name
   }
   dataArr.push(fullName)
 })
 .on("end", () => {
   console.log(dataArr)
 });

In the callback function of the data event, we are now creating a new object and adding the first name and last name by referencing the data from each row. When you run the code, the following is what is logged to the console.

[
 { firstName: 'Isabeau', lastName: 'Bramwell' },
 { firstName: 'Gipsy', lastName: 'Frickey' },
 { firstName: 'Hunt', lastName: 'Rabbage' },
 { firstName: 'Wilden', lastName: 'Wolters' },
 { firstName: 'Kalie', lastName: 'Hayley' }
]

 

Conclusion

To recap, a CSV file is a text file used to store tabular data as comma-separated values. They are mainly used for storing and sharing large amounts of data. In this article, you learned how to parse CSV data in three ways. The first method reads the whole string then splits it into rows and columns. While this method works, it exposes our program to out-of-memory issues. The second method solves this issue by reading the CSV line by line. The last method takes advantage of the csv-parser module, an npm package that makes reading and writing CSV data easy.

 

Learn More

To learn more about parsing CSV files in NodeJS:

• Check out the csv-parser module documentation on Github.
• Learn more about the fs module and readline module.