Amazon s3 upload multiple files

Mitch Dempsey Mitch Dempsey 3 3 silver badges 10 10 bronze badges. To use versioning, you need to turn it on for your S3 bucket. Tom Andersen Tom Andersen 1 1 silver badge 5 5 bronze badges. Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Podcast Helping communities build their own LTE networks.

Podcast Making Agile work for data science. Featured on Meta. New post summary designs on greatest hits now, everywhere else eventually. Related 1. Tomasz Kowalczyk Lucas Romano Lucas Romano 4 4 bronze badges.

Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Podcast Helping communities build their own LTE networks. Podcast Making Agile work for data science. Featured on Meta. New post summary designs on greatest hits now, everywhere else eventually.

Visit chat. Active 2 years, 1 month ago. Viewed 2k times. Any thoughts would be much appreciated! Improve this question. Sotiris Kaniras Sotiris Kaniras 1 1 silver badge 10 10 bronze badges. This may be worth reading: serverfault. Add a comment. Active Oldest Votes. Improve this answer. Stennie Stennie 1, 7 7 silver badges 12 12 bronze badges. Thanks for your very well explained answer! Do you think it's a feasible tool for 45gb of data? S3cmd supports recursive file uploads but I don't see any options for adjusting concurrency settings.

As far as feasibility goes, I don't foresee any obvious issues as long as you choose appropriate settings. I would test with a smaller set of files to find the best concurrency options since these could be limited by resources on your source instance.

Given your earlier question about Parse, I'm also not sure if you've fully described what you are trying to solve. This question only mentions uploading images, but if this is one step of a migration from GridFS to S3 storage you probably want to rewrite the image paths in MongoDB as well.

However, I'm answering the question as posed here :.