Retrieving subfolders names in S3 bucket from boto3

👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!

Using boto3, I can access my AWS S3 bucket:

s3 = boto3.resource("s3")
bucket = s3.Bucket("my-bucket-name")

Now, the bucket contains folder first-level, which itself contains several sub-folders named with a timestamp, for instance 1456753904534. I need to know the name of these sub-folders for another job I"m doing and I wonder whether I could have boto3 retrieve those for me.

So I tried:

objs = bucket.meta.client.list_objects(Bucket="my-bucket-name")

which gives a dictionary, whose key "Contents" gives me all the third-level files instead of the second-level timestamp directories, in fact I get a list containing things as

{u"ETag": ""etag"", u"Key": first-level/1456753904534/part-00014", u"LastModified": datetime.datetime(2016, 2, 29, 13, 52, 24, tzinfo=tzutc()),
u"Owner": {u"DisplayName": "owner", u"ID": "id"},
u"Size": size, u"StorageClass": "storageclass"}

you can see that the specific files, in this case part-00014 are retrieved, while I"d like to get the name of the directory alone. In principle I could strip out the directory name from all the paths but it"s ugly and expensive to retrieve everything at third level to get the second level!

I also tried something reported here:

for o in bucket.objects.filter(Delimiter="/"):
    print(o.key)

but I do not get the folders at the desired level.

Is there a way to solve this?

👻 Read also: what is the best laptop for engineering students?

Retrieving subfolders names in S3 bucket from boto3 __del__: Questions

import time
time.sleep(5)   # Delays for 5 seconds. You can also use a float value.

import time
while True:
    print("This prints once a minute.")
    time.sleep(60) # Delay for 1 minute (60 seconds).

from time import sleep
sleep(0.1) # Time in seconds

Retrieving subfolders names in S3 bucket from boto3 __del__: Questions

We hope this article has helped you to resolve the problem. Apart from Retrieving subfolders names in S3 bucket from boto3, check other __del__-related topics.

Want to excel in Python? See our review of the best Python online courses 2023. If you are interested in Data Science, check also how to learn programming in R.

By the way, this material is also available in other languages:

• Italiano Retrieving subfolders names in S3 bucket from boto3
• Deutsch Retrieving subfolders names in S3 bucket from boto3
• Français Retrieving subfolders names in S3 bucket from boto3
• Español Retrieving subfolders names in S3 bucket from boto3
• Türk Retrieving subfolders names in S3 bucket from boto3
• Русский Retrieving subfolders names in S3 bucket from boto3
• Português Retrieving subfolders names in S3 bucket from boto3
• Polski Retrieving subfolders names in S3 bucket from boto3
• Nederlandse Retrieving subfolders names in S3 bucket from boto3
• 中文 Retrieving subfolders names in S3 bucket from boto3
• 한국어 Retrieving subfolders names in S3 bucket from boto3
• 日本語 Retrieving subfolders names in S3 bucket from boto3
• हिन्दी Retrieving subfolders names in S3 bucket from boto3