Amazon S3 listing "directories"

You can specify any directory hierarchy that lists buckets. When prefix is ​​set to "" or "/", it indicates the first-level directory of the bucket. When setting a specific subdirectory, the list of keys of the subdirectory is displayed.

Reference to Charles Menguy's answer in 'Amazon S3 listing “directories”', and added some modifications.

  public List<String> listKeysInBucket(String bucketName, String prefix) {
    Boolean isTopLevel = false;
    String delimiter = "/";
    if(prefix == "" || prefix == "/") {
      isTopLevel = true;
    }
    if (!prefix.endsWith(delimiter)) {
      prefix += delimiter;
    }

    ListObjectsRequest listObjectsRequest = null;
    if (isTopLevel) {
      listObjectsRequest =
          new ListObjectsRequest().withBucketName(bucketName).withDelimiter(delimiter);
    } else {
      listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withPrefix(prefix)
          .withDelimiter(delimiter);
    }
    ObjectListing objects = s3Client.listObjects(listObjectsRequest);
    return objects.getCommonPrefixes();
  }

This below code worked for me to list all directories in s3.

private static String bucket_name = "";
private static String secret_key = "";
private static String access_key = "";
private static String Regions region = Regions.SELECT_REGION;

public static void main(String[] args) {
        System.out.println(listKeysInBucket(bucket_name, "/"));
    }

    public static List<String> listKeysInBucket(String bucketName, String prefix) {
        boolean isTopLevel = false;
        String delimiter = "/";
        if (prefix.equals("") || prefix.equals(delimiter)) {
            isTopLevel = true;
        }
        if (!prefix.endsWith(delimiter)) {
            prefix += delimiter;
        }

        ListObjectsRequest listObjectsRequest = null;
        if (isTopLevel) {
            listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withDelimiter(delimiter);
        } else {
            listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withPrefix(prefix)
                    .withDelimiter(delimiter);
        }

        ObjectListing objects = s3Client().listObjects(listObjectsRequest);
        return objects.getCommonPrefixes();
    }

    public static AmazonS3 s3Client() {
        AWSCredentials s3Configs = new BasicAWSCredentials(access_key,secret_key);
        return AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(s3Configs)).withRegion(region )
                .build();
    }

Where you have keys that have no content S3 considers them "Common Prefixes":

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectListing.html#getCommonPrefixes%28%29

public List getCommonPrefixes()

Gets the common prefixes included in this object listing. Common prefixes are only present if a delimiter was specified in the original request.

Each common prefix represents a set of keys in the S3 bucket that have been condensed and omitted from the object summary results. This allows applications to organize and browse their keys hierarchically, similar to how a file system organizes files into directories.

For example, consider a bucket that contains the following keys:

"foo/bar/baz"
"foo/bar/bash"
"foo/bar/bang"
"foo/boo"

If calling listObjects with the prefix="foo/" and the delimiter="/" on this bucket, the returned S3ObjectListing will contain one entry in the common prefixes list ("foo/bar/") and none of the keys beginning with that common prefix will be included in the object summaries list.

Returns: The list of common prefixes included in this object listing, which might be an empty list if no common prefixes were found.

Tags:

Java

Amazon S3