Aspera on Demand support for S3 in different regions

Overview

This article describes some basic requirements for Aspera On Demand when connecting and transferring data to S3.

 

Details

Aspera on Demand provides a high speed transfer system for organizations and users to upload and download data to the Amazon cloud.  Aspera on Demand supports various storage options, such as EBS, EFS, Ephemeral, Glacier and S3.  This article discusses the region requirements when using Aspera to connect to an S3 bucket.

 

Requirements

Aspera on Demand servers (Server on Demand, Application Platform on Demand, Shares on Demand, Faspex on Demand) must reside in the same region as the S3 bucket you are attempting to connect to.   The reason for this is performance related.  Aspera on Demand leverages S3's native HTTP APIs when writing to or reading from S3.  These APIs are the only option for connecting with S3.  The APIs provide sufficient performance when connecting over the local area network (e.g. in the same AWS region), however, when using these HTTP(s) APIs over the WAN, there is a significant performance degradation.  This performance degradation is noticeable by anyone who is attempting to move large volumes of data up to S3.

For this reason, the Aspera Server system, explicitly prevents you from configuring your Aspera server to connect via the APIs to an bucket located in a different S3 region.  The detection of the servers region and the buckets region are done on transfer session initiation.  If the regions do not match, the error is logged on the Aspera server in the trapd logs (/opt/aspera/var/log/trapd/aspera-trapd.log).

The Aspera On Demand Entitlement allows you to run Aspera servers in multiple regions for this very reason.

 

Conclusion

If your data transfer workflow requires that you move data to multiple different AWS regions, then it is required that you run at least one Aspera on Demand Servers in each region where your S3 buckets are located.  The following diagram depicts a multi-region architecture that can be deployed to deliver content to multiple regions.   The subsequent diagram, depicts an architecture that is not allowed, and will create error messages.

 

S3_region_support.png

 

Figure 1:  A successful deployment of Aspera in multiple regions.

 

S3_not_supported.png

 

Figure 2: An incorrect configuration of Aspera that will not work.

 

 

 

 

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk