Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Renaming files using Amzon CLI script object

Hi !
Can anyone help solve this ?

I have an S3 bucket with two folders. S3://BKP and S3://DIMS.

I have to rename files in a way as shown below

DIM1_TOP*.TXT to DIM1_TOP.TXT
DIM1_XOT*.TXT to DIM1_XOT.TXT
DIM2_TOP*.TXT to DIM2_TOP.TXT
DIM2_XOT*.TXT to DIM2_XOT.TXT

Basically I want to remove anything after TOP or XOT.
One condition is I may not always receive all the 4 files.
for example I do not have DIM2_XOT*.TXT. But still my code should not fail or it simply ignore the error and rename those files whichever are available.

3 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi Nachiket,

One option is to use a file-iterator to iterate over each file and take necessary action using the AWS CLI via the Bash or Python component. This way, you are dealing with one file at a time and you can implement your logic in a Bash/Python/child-job which may receive the single file to act on.

You may also attach a S3 Put Component to the file-iterator. It can copy files between S3 locations.

Hope that helps

Best
Kalyan


Nachiket Mehendale —

Hi Kalyan, Thank you.

Let me draft a mail to the support desk as I require some more clarity on this part.


Matillion Agent  

Kalyan Arangam —

Hi Nachiket,

Please do get back to us if you need further advice on this workflow.

Best
Kalyan

Post Your Community Answer

To add an answer please login