How do I compute a digest for a given blob in big size, e.g. 5GB?

I believe the right answer would be to stream the file content instead of reading whole file in memory at once. Blob allows to read a file as a stream: https://developer.mozilla.org/en-US/docs/Web/API/Blob/stream

Now the problem is that Web Cryptography API you're using doesn't support streams or incremental hashing. There is a long (and quite old) discussion about that with no clear outcome: https://github.com/w3c/webcrypto/issues/73 .

I would suggest using some 3rd party library that supports incremental hashing. E.g. https://github.com/Caligatio/jsSHA

The resulting code could look like

async function calcDigest() {
    const reader = finput.files[0].stream().getReader()
    const shaObj = new jsSHA("SHA-256", "ARRAYBUFFER")

    while (true) {
        const {done, value} = await reader.read()
        if (done) break
        shaObj.update(value)
    }

    console.log(shaObj.getHash("HEX"))
}

Tags:

Javascript