Search nested subdirectory in S3 bucket

Avinash Dalvi
1 min readJun 19, 2020

I am explaining about search nested subdirectory is exist in S3 bucket or not. Created AWS lambda code in python using boto3 to find existence of sub directory.

client.list_objects(Bucket=_BUCKET_NAME, Prefix=_PREFIX)

this function gives list of all content exist in bucket along with path. It will be easy to trace it out.

import boto3

client = boto3.client('s3')
bucket_name = "bucket_name"
prefix = ""

s3 = boto3.client("s3")

result = client.list_objects(Bucket=bucket_name, Delimiter='/')
for obj in result.get('CommonPrefixes'):
prefix = obj.get('Prefix')
file_list = ListFiles(client,bucket_name,prefix)
for file in file_list:
if "processed/files" in file:
print("Found",file)

def ListFiles(client, bucket_name, prefix):
_BUCKET_NAME = bucket_name
_PREFIX = prefix
"""List files in specific S3 URL"""
response = client.list_objects(Bucket=_BUCKET_NAME, Prefix=_PREFIX)

for content in response.get('Contents', []):
#print(content)
yield content.get('Key')

Github gist : https://gist.github.com/aviboy2006/bf3feb8828d2fb311ffe22b750b2b297

Stackoverflow : https://stackoverflow.com/questions/62158664/search-in-each-of-the-s3-bucket-and-see-if-the-given-folder-exists/62160218#62160218

Originally published at https://dev.to on June 19, 2020.

--

--

Avinash Dalvi

Enthusiastic learner, Full Stack Developer, Techno Savvy, Traveller, Out of Box thinker, Agile Lover, Problem Solver, Blogger