Node.js has some built-in modules in order to download files. Using those modules, it is easy to download a file (image, text, pdf, or any other type) and save it to the filesystem.
https is part of the core Node.js modules that allow sending HTTPS requests. On the other hand, the fs module gives access to the local filesystem. By using these two modules together, it is possible to send a GET request and save the response stream in a new file.
In simple words, you can download files from publicly available sources just with a few lines of code.
These are the steps to download a file using Node.js
- Send a GET request to the URL from which you want to download the file using
https.get()method. - Create a writeable stream with the
fs.createWriteStream()method. It needs the location where you want to save your downloaded file. - Call the
pipe()method on the response object that you receive inside the callback function ofhttps.get()method and pass the writeable stream as its argument. - Close the writeable stream with the
close()method when the writeable stream emits thefinishevent.
These are the steps you need to follow when you download files just using NodeJS modules. There are some external packages that you can use to make the process more simple if you want.
Also Read: Easy Ways to Download Any File From Node.js Server Using Express
In this article, I will show you both ways so that you are able to use Node.js modules or external packages in order to download a file.
Let's get started...
Download File Using Node.js Core Modules (HTTPS)
Step 1: You have to send a GET request to the target URL using https.get() method. This method accepts 2 arguments, an URL and a callback function.
const https = require('https');
const url = 'https://www.webmound.com/uploads/26/banner.jpg';
https.get(url, (res) => {
// Create a writeable stream
// Save your file from the response
}).on('error', (err) => {
// handle error
console.log(err);
});
First, I am importing the https module. Then I am calling the https.get() method with the URL and the callback function.
In the first argument of this callback function, you will get the response object. This response object will contain the file data as a readable stream.
You can handle errors by listening to the error event using the on() method.
Step 2: You have to create a writeable stream with the fs.createWriteStream() method from the fs module. This method requires a path with the name of the file where you want to save it.
const fs = require('fs');
const https = require('https');
const url = 'https://www.webmound.com/uploads/26/banner.jpg';
https.get(url, (res) => {
const imagePath = './img/my-banner.jpg';
const stream = fs.createWriteStream(imagePath);
}).on('error', (err) => {
// handle error
console.log(err);
});
I want to save the download image inside an img folder with the my-banner.jpg name. So, I am setting the path accordingly in the imagePath variable.
Note: You must create the folder manually mentioned in the image path. Because
fs.createWriteStream()method will not create a folder. And make sure you use the same file extension as the url.
When I call the fs.createWriteStream() method, I will pass the path to it. This method will return a writeable stream. I am storing that in the stream variable.
Step 3: Now call the pipe() method on the response object to write the response data to the writeable stream.
const fs = require('fs');
const https = require('https');
const url = 'https://www.webmound.com/uploads/26/banner.jpg';
https.get(url, (res) => {
const imagePath = './img/my-banner.jpg';
const stream = fs.createWriteStream(imagePath);
res.pipe(stream);
}).on('error', (err) => {
// handle error
console.log(err);
});
I am calling res.pipe() method and passing the stream variable to it.
Step 4: Finally, you will listen to the finish event by calling the on() method on the stream variable. And close the stream when it is done.
const fs = require('fs');
const https = require('https');
const url = 'https://www.webmound.com/uploads/26/banner.jpg';
https.get(url, (res) => {
const imagePath = './img/my-banner.jpg';
const stream = fs.createWriteStream(imagePath);
res.pipe(stream);
stream.on('finish', () => {
stream.close();
console.log('Image downloaded');
});
}).on('error', (err) => {
// handle error
console.log(err);
});
I am calling stream.on() method. In the first argument, you have to give the name of an event. Here, I want to listen to the finish event.
Inside the event callback function, I will use stream.close() method to close the stream. This callback function will only be executed when the downloading is completed.
This is the complete process of downloading a file just using Node.js modules.
Now save the above code in a JavaScript file and execute it with the node command. I am saving the code in a main.js file. To execute the file, run the following command:
node main.js
After executing the command, I will see a new image file with the name my-banner.jpg inside the img folder.
In this example, I am downloading an image file. But you are not limited to images. You can download any type of file with this method.
Passing Command-line Arguments in Node.js
If we hard-code the source URL in the script, it becomes less reusable. Whenever you want to download a file from a different URL, you need to change the code.
But we can make our Node.js code reusable for downloading files from any source without changing anything in our script.
We can pass the file URL from where we want to download the file and the destination path where we want to store the downloaded file as command-line arguments in Nodejs.
node main.js https://www.webmound.com/uploads/26/banner.jpg ./img/my-banner.jpg
When we execute our JavaScript files using the node command, we will pass the URL of the file and folder path where we want to save that file after the name of our JavaScript file.
Here, I am executing my main.js file but also passing my image URL and my folder path. Now I have to access these values inside my Node.js script.
How can you access the command line arguments in Node.js?
When we execute the node command, in our Node.js code we get access to process.argv automatically. It's an array of strings.
In our command, the URL and folder path is present in the 3rd and 4th positions respectively. So, they are available in the process.argv array at index 2 and 3 respectively.
const url = process.argv[2];
cosnt folder = process.argv[3];
Now, you can use these dynamic values in your Node.js code without hard-coding them. In this way, you will be able to use the same script for downloading files from any source just passing them in the command line.
Our Node.js code will look like this with the dynamic value for source URL and file destination path:
const fs = require('fs');
const https = require('https');
const url = process.argv[2];
https.get(url, (res) => {
const imagePath = process.argv[3];
const stream = fs.createWriteStream(imagePath);
res.pipe(stream);
stream.on('finish', () => {
stream.close();
console.log('Image downloaded');
});
}).on('error', (err) => {
// handle error
console.log(err);
});
I have changed the values of url and imagePath variables. Everything else is the same as previous. Now whenever you execute this file, you just have to pass these two values.
Also Read: How to Get All Files in Directories (Recursively) with Node.js
How to Download Multiple Files Using Node.js Core Modules
until now we were downloading a single file at a time. But what if you need to download multiple files. Downloading each file separately is really a pain especially if you want to download many files.
But we can make this process simple by downloading multiple files from different sources automatically. The technique is the same. We will use Node.js core modules as we have seen in our previous sections.
const fs = require('fs');
const https = require('https');
const download = (url, path) => {
https.get(url, (res) => {
const stream = fs.createWriteStream(path);
res.pipe(stream);
stream.on('finish', () => {
stream.close();
console.log('Image downloaded');
});
}).on('error', (err) => {
// handle error
console.log(err);
});
};
const urls = [
'https://www.webmound.com/uploads/26/default.jpg',
'https://www.webmound.com/uploads/26/banner.jpg',
];
urls.forEach((url) => {
const fileName = url.split('/').pop();
download(url, `./img/${fileName}`);
});
I am creating a function called download and it accepts url and path arguments. Inside this function, we will use the same process of downloading a file.
The only difference is that we will use the argument url value in https.get() method and path value in fs.createWriteStream() method.
Now you can call this function as many times as you want with these two arguments.
Next, I have created an array of URLs. This array will contain all the source URLs from where you want to download your files. You can add as many URLs as you want to this array.
Finally, I will loop over this urls array using the forEach() method. Inside the callback function for each file, I need to define a filename.
Also Read: Perfect Ways to Serve Static Files in Express and Node.js
I am getting the filename by splitting the URL on the "/" using the split() method. And by adding the pop() method, I am getting the last item from the array created by the split() method.
In this example, I am getting default.jpg and banner.jpg as the filenames.
You can learn more about the split() method here and the pop() method here.
I am calling the download function with the url and the folder path where I want to store my downloaded files.
That's it. When you execute this code, it will download multiple files using the list of URLs from that array. And for that, you just have to run this code once.
Note: For downlaoding multiple files, I prefer to use external NPM package for some reasons. I will show you how and why you should use it for multiple files in the next section.
Download File Using "download" Package
If you don't want to send HTTPS requests and stream files manually using the Node.js core module, you have the option to use an external package.
Initialize your NodeJS project using the following command. It will create a package.json file. After that, you can install your external packages for this project.
npm init -y
Also Read: Best Setup to Use TypeScript with Node.js and Express Project
You can use the "download" package to download files in NodeJS. Install this package in your project with the following command.
npm install download
I will import and use this package in my main.js file. This package gives a function that returns a promise.
You have to pass 2 arguments to this function an URL from which you want to download your file and the folder path where you want to store your downloaded file.
Note: You don't need to specify the name of your donwloaded file. This package will automatically name the file according to the source URL.
const download = require('download');
const url = 'https://www.webmound.com/uploads/26/banner.jpg';
(async () => {
await download(url, './img');
console.log('Image downloaded');
})();
Here I am using a self-executing function with the async keyword. Inside it, I can call the download function with the 2 arguments. I will use await keyword in front of it because it returns a promise.
With these few lines of code, you can download a file from an URL.
Download Multiple Files Using "download" Package in Node.js
You can also download multiple files using the "download" package. For downloading multiple files, I really recommend you to use this package instead of writing your own code.
Because this package download files asynchronously. Therefore, it will take less time compared to the synchronous method.
There is a big difference between the synchronous and asynchronous methods in terms of performance.
const download = require('download');
(async () => {
const urls = [
'https://www.webmound.com/uploads/26/default.jpg',
'https://www.webmound.com/uploads/26/banner.jpg',
];
await Promise.all(urls.map((url) => download(url, './img')));
console.log('Images downloaded');
})();
I am defining an array of URLs as the source. I am using the Promise.all() method to handle multiple promises at once.
By calling the map() method on urls array, I am creating another array of promises from those URLs passing it to the Promise.all() method.
When you execute the above code, it will download and save both default.jpg and banner.jpg files in the ./img folder with the same filenames.
Conclusion
You can use Node.js core modules or an external package to download files. Each technique has its pros and cons.
If you use core modules then you don't have to install any additional dependency package but you have to set up everything manually.
On the other hand, by installing an external package you can download a file easily with a few lines of code. This package also returns a promise. So, for downloading multiple files at once you will get better performance.
Now it's totally up to you which technique you want to use.
In this article, I have shown you all the possible ways to download a single file or multiple files using Nodejs.
I hope it was helpful for you. Thank you so much for reading. Happy coding!
