Manually Setting up the Default Shares S3 Docroot

What are Nodes, API Users, S3 Buckets, and S3 Docroots (and how are they related)?

An API user is a system user with a docroot that is used for API level access, like listing of files and initiating/authenticating transfers.

A Node in Shares is a System with Aspera Connect Server (Enterprise Server that is Web Plugin enabled) and a docroot on the system as represented by an API User. More accurately, a Shares Node is a system that Shares shares-out as represented by an API User.

S3 Buckets are discrete storage containers on the S3 object-store. Buckets are not like a file-system where there is a root-level or namespace. Each is a separate container. As such, there is no way to create a single Aspera Docroot that encapsulates multiple buckets.

S3 Docroots map the root-level of a transfer destination/source to a single S3 bucket.

Since the API User views storage as a docroot, and an S3 docroot only maps to a single bucket, having Shares view multiple S3 buckets requires unique/individual API users (and nodes in Shares) for each bucket.

Manually Setting up the Default Shares S3 Docroot:

To simplfy the Shares OnDemand initial deployment, 2 API users are created: xfer and xfer2. BOth API Users have a default password of the instance ID of the running EC2 instance. One user xfer has a docroot of the local ephemeral storage. This is a high speed local storage system that does not persist across instance reboots.

The second API user xfer2 also has a docroot set to the ephemeral storage, but it is intended to be changed to an S3 location.

The S3 location is given in the form of a [URL] (http://en.wikipedia.org/wiki/Uniform_resource_locator). In general, the format is:

s3://AWSAPIUSER:AWSAPIKEY@s3.amazonaws.com/BUCKETNAME

Where

  • AWSAPIUSER is the AccessKeyID found on the AWS account/IAM user (https://forums.aws.amazon.com/thread.jspa?threadID=49738);
  • AWSAPIKEY is the SecretAccessKey found on the AWS account/IAM user (https://forums.aws.amazon.com/thread.jspa?threadID=49738);
  • BUCKETNAME is the S3 bucket for the data.

Please note that the AWSAPIKEY/SecretAccessKey can have special characters that do not conform to the URL syntax. These need to be escaped using [percent encoding] (http://en.wikipedia.org/wiki/Percent-encoding).

Once a correct S3 URL is obtained, it can be set as the docroot for the xfer2 user.

  1. This can be done through manually editing the configuration file for Aspera (/opt/aspera/etc/aspera.conf), the built-in Console can be used to edit the xfer2 docroot, or asconfigurator can be used. Assuming a root-level terminal to the EC2 instance already exists, the simple asconfigurator command is:

    # asconfigurator -F "set_user_data;user_name,xfer2;absolute,s3://AWSAPIUSER:AWSAPIKEY@s3.amazonaws.com/BUCKETNAME"
  2. Then the asperanoded service needs to be restarted for the docroot to take effect.

    # service asperanoded restart
  3. Log into Shares as Admin, and add a new Node by clicking on the ‘+’ symbol next to the Node menu heading. S3-Step1-LoggedIn.png

  4. Add the following fields, as depicted in the screenshot below, including the API password. (the default password is the instance ID.)S3-Step2-NewNode.png
  5. Browse the newly added node (S3 Storage) by clicking on it, and confirm that you can now see the contents of your S3 bucket. S3-Step3-Result.png
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk