This was originally posted on blog.emccode.com
--------------
“Object” is common place in new applications and many vendors are offering this new generation of object-based storage. The one thing that has made object storage so successful is the lowest common denominator of S3. You may know it as Simple Storage Service from AWS, but Amazon has followed a simple REST based approach that many vendors just refer to as “S3 Storage”. There are many S3 compatible storage vendors such as Cleversafe, RiakCS, Ceph, vCloud Air Object Storage, and, of course, EMC ViPR and EMC ECS. Each vendor uses a similar API structure to follow Amazon’s S3 service.
Many people start their object storage journey using AWS, but at some point must look at other alternatives for cost and/or security. The piece that was missing is the ability to migrate objects from one S3 compatible service to another. This is where s3motion plays that crucial role.
s3motion is a combination of both a CLI tool and REST based microservice that can upload/download/copy/migrate objects between local filesystems and S3 storage or between S3 compatible storage systems.
Some common use cases:
- a command line tool to upload/download objects.
- move or copy individual objects at scheduled intervals between buckets.
- copy an object from one S3 service to many after it has experienced a time-stamp change.
- migrate between services. copy an entire bucket, then change your application to point to new S3 storage provider
s3motion can be installed through node package manager with:
npm install s3motion -g
or as a Docker container with:
docker run -d -p 8080:8080 emccode/s3motion
All of the operations are documented for both CLI and REST.
Watch s3motion in the video below to see how it works. s3motion is an open source project.