How can I get the size of an Amazon S3 bucket?
Solution 1:
This can now be done trivially with just the official AWS command line client:
aws s3 ls --summarize --human-readable --recursive s3://bucket-name/
Official Documentation: AWS CLI Command Reference (version 2)
This also accepts path prefixes if you don't want to count the entire bucket:
aws s3 ls --summarize --human-readable --recursive s3://bucket-name/directory
Solution 2:
The AWS CLI now supports the --query parameter which takes a JMESPath expressions.
This means you can sum the size values given by list-objects using sum(Contents[].Size) and count like length(Contents[]).
This can be be run using the official AWS CLI as below and was introduced in Feb 2014
aws s3api list-objects --bucket BUCKETNAME --output json --query "[sum(Contents[].Size), length(Contents[])]"
Solution 3:
AWS Console:
As of 28th of July 2015 you can get this information via CloudWatch. If you want a GUI, go to the CloudWatch console: (Choose Region > ) Metrics > S3
AWS CLI Command:
This is much quicker than some of the other commands posted here, as it does not query the size of each file individually to calculate the sum.
aws cloudwatch get-metric-statistics --namespace AWS/S3 --start-time 2015-07-15T10:00:00 --end-time 2015-07-31T01:00:00 --period 86400 --statistics Average --region eu-west-1 --metric-name BucketSizeBytes --dimensions Name=BucketName,Value=toukakoukan.com Name=StorageType,Value=StandardStorage
Important: You must specify both StorageType and BucketName in the dimensions argument otherwise you will get no results. All you need to change is the --start-date, --end-time, and Value=toukakoukan.com.
Here's a bash script you can use to avoid having to specify --start-date and --end-time manually.
#!/bin/bash
bucket=$1
region=$2
now=$(date +%s)
aws cloudwatch get-metric-statistics --namespace AWS/S3 --start-time "$(echo "$now - 86400" | bc)" --end-time "$now" --period 86400 --statistics Average --region $region --metric-name BucketSizeBytes --dimensions Name=BucketName,Value="$bucket" Name=StorageType,Value=StandardStorage
Solution 4:
s3cmd can do this :
s3cmd du s3://bucket-name
Solution 5:
If you download a usage report, you can graph the daily values for the TimedStorage-ByteHrs field.
If you want that number in GiB, just divide by 1024 * 1024 * 1024 * 24 (that's GiB-hours for a 24-hour cycle). If you want the number in bytes, just divide by 24 and graph away.