Migrate a bucket from another provider to Hetzner Object Storage

Last change on 2024-12-06 • Created on 2024-12-06 • ID: ST-4C94A

You can migrate data from a bucket at another provider to a Bucket in Hetzner Object Storage by following the steps below.

If you want to migrate from AWS S3, note that they offer "Free data transfer out to internet when moving out of AWS".

The commands depend on the S3-compatible tool you're using. This getting started explains each step with example commands for the MinIO Client, RCLONE, S3cmd, and the AWS CLI.

  1. Migrate data

    Note that rclone, s3cmd and aws s3 will NOT copy custom metadata. If any of your objects have custom metadata, it will be lost on the new Bucket when using these clients. If you need the custom metadata, you consider using the MinIO Client.

    • MinIO Client

      mc mirror <alias_name>/<source_bucket> <alias_name>/<destination_bucket>

    • rclone

      rclone sync <remote_name>:<source_bucket> <remote_name>:<destination_bucket>

    • S3cmd

      You cannot specify two different configuration files at once. Instead, you have to temporarily store the data on your local device and sync the data to your Bucket from there.

      mkdir migrate-buckets && cd migrate-buckets
      s3cmd sync -c ~/.conf-1 s3://<source_bucket> .
      s3cmd sync -c ~/.conf-2 . s3://<destination_bucket>

    • AWS CLI

      You cannot specify two different profiles at once. Instead, you have to temporarily store the data on your local device and sync the data to your Bucket from there.

      mkdir migrate-buckets && cd migrate-buckets
      aws s3 sync --profile <profile_name> s3://<source_bucket> .
      aws s3 sync --profile <profile_name> . s3://<destination_bucket>

  1. View the Bucket content

    • MinIO Client

      mc ls <alias_name>/<destination_bucket>

    • rclone

      rclone ls <remote_name>:<destination_bucket>

    • S3cmd

      s3cmd ls -c ~/.s3cfg-provider-2 s3://<destination_bucket>

    • AWS CLI

      aws s3 ls --profile <profile_name> s3://<destination_bucket>

Your data should now also be available in your Hetzner Object Storage Bucket.