GZIP Compression on static Amazon S3 files

Files should be compressed before being uploaded to Amazon S3.

For some examples, see:

• Serving Compressed (gzipped) Static Files from Amazon S3 or Cloudfront
• How to: Gzip compression of CSS and JS files on S3 with s3cmd

If you use CloudFront in front of your S3 bucket, there is no need to manually compress HTML ressources (CloudFront will compress them on-the-fly). Please note CloudFront only compress in gzip (no deflate, brotli) and only CSS / JS / HTML (based on content-type). See https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html#compressed-content-cloudfront-file-types . To make it works, you have to forward some http headers from CloudFront to S3 (see doc).

If your S3 bucket have resources not supported by Cloudfront (generic "binary/octet-stream" mime type, like "hdr" texture or "nds" ROM), you need to compress them by yourself before uploading to S3, then set the "content-encoding" http meta on the resource. Note that only browsers supporting the gz encoding will be able to download and decompress the file.

If you don't want to compress the file one-by-one by the hand, you can use a Lambda function

• triggered on each PUT of an object (a file) in the bucket
• if the file is not already compressed and if compression is usefull, then replace the original uploaded file with the compressed version
• set http headers content-encoding to gzip

I wrote a GIST for this, it can inspire you to create your own process. See https://gist.github.com/psa-jforestier/1c74330df8e0d1fd6028e75e210e5042

And dont forget to invalidate (=purge) Cloudfront to apply your change.

If you simply want to gzip the existing files in your S3 bucket, you can write a Lambda function for it. Read the files into a buffer and then use the gzip library to compress them and re-upload to S3. Something like this should work:

 gzipped_content = gzip.compress(f_in.read())
                destinationbucket.upload_fileobj(io.BytesIO(gzipped_content),
                                                        final_file_path,
                                                        ExtraArgs={"ContentType": "text/plain"}
                                                )

There's a full tutorial here: https://medium.com/p/f7bccf0099c9