Node.js and Request - limiting the file size of downloaded file

I had a similar issue. I use now fetch to limit download size.

const response = await fetch(url, {
    method: 'GET',t
    size: 5000000, // maximum response body size in bytes, 5000000 = 5MB 
}).catch(e => { throw e })

I would actually use both methods you've discussed: check the content-legnth header, and watch the data stream to make sure it doesn't exceed your limit.

To do this I'd first make a HEAD request to the URL to see if the content-length header is available. If it's larger than your limit, you can stop right there. If it doesn't exist or it's smaller than your limit, make the actual GET request. Since a HEAD request will only return the headers and no actual content, this will help weed out large files with valid content-lengths quickly.

Next, make the actual GET request and watch your incoming data size to make sure that it doesn't exceed your limit (this can be done with the request module; see below). You'll want to do this regardless of if the HEAD request found a content-length header, as a sanity check (the server could be lying about the content-length).

Something like this:

var maxSize = 10485760;

request({
    url: url,
    method: "HEAD"
}, function(err, headRes) {
    var size = headRes.headers['content-length'];
    if (size > maxSize) {
        console.log('Resource size exceeds limit (' + size + ')');
    } else {
        var file = fs.createWriteStream(filename),
            size = 0;

        var res = request({ url: url });

        res.on('data', function(data) {
            size += data.length;

            if (size > maxSize) {
                console.log('Resource stream exceeded limit (' + size + ')');

                res.abort(); // Abort the response (close and cleanup the stream)
                fs.unlink(filename); // Delete the file we were downloading the data to
            }
        }).pipe(file);
    }
});

The trick to watching the incoming data size using the request module is to bind to the data event on the response (like you were thinking about doing using the http module) before you start piping it to your file stream. If the data size exceeds your maximum file size, call the response's abort() method.