Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Jython boto3 delete all files in S3 sub-folders

Hello

I have bucket with several folders. One of this folders is ETLWork folders.
ETLWork folder has several sub-folders like external_table1, external_table2 ...
How can I delete all files in sub-folders?

Thank you
Dmitriy Burtsev

6 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi Dmitriy,

I prefer using the AWS CLI commands via the BASH component to achieve this. Here’s a single line to do what you are after -

aws s3 rm s3://mybucket/myfolder/ETLWork/ —recursive

More information on this at the following link – https://docs.aws.amazon.com/cli/latest/reference/s3/rm.html
Needless to say, please test this on a safe location to gain some confidence :)

If you are keen to go down the Python+boto route, the following link seem to have an example for the same.

https://stackoverflow.com/questions/43944190/how-to-delete-folder-and-its-content-in-a-aws-bucket-using-boto3?rq=1

Hope that helps.

Best
Kalyan


Dmitriy (Consultant) Burtsev —

Hi

I already have a Jython script. I don't want to call BASH script in the middle of Python script.
Your example is "How to delete folder and its content in a AWS bucket using boto3". I don't want to delete folder.
Thank you,
Dmitriy Burtsev


Edward Hunter —

Hey Dmitriy;

Try this.

import boto
def delete_s3_recursive(bucketname, keyprefix):
s3 = boto_connect_s3()
bucket = s3.get_bucket(bucketname,validate=False)
bucketListResultSet = bucket.list(prefix=keyprefix)
return bucket.delete_keys([key.name for key in bucketListResultSet])

Given the two parameters, bucketname and keyprefix (the filename mask for targeted files), this should do what you are hoping. Bear in mind your credentials must support the activity for the given bucket/files.

Good Luck,

Edward Hunter


Dmitriy (Consultant) Burtsev —

Hello
I am using boto3, not boto
"AttributeError: 's3.ServiceResource' object has no attribute 'get_bucket'"

How can I do this with boto3?

Thank you,
Dmitriy Burtsev


Dmitriy (Consultant) Burtsev —


This boto3 code delete all files and folders. I would like to delete only files.
s3 = boto3.resource('s3')

BucketW = s3.Bucket('BUCKET')
BucketW.objects.filter(Prefix = 'ETLWork').delete()


Dmitriy (Consultant) Burtsev —

BucketW = s3.Bucket(BUCKET)
for obj in BucketW.objects.filter(Prefix = ETLWork):
if '.txt' in obj.key:
obj.delete()

Post Your Community Answer

To add an answer please login