How to enable your Faspex On Demand to use S3 as a remote source

 

Summary

Faspex On Demand 3.0 now supports the ability to use S3 storage as a source for selecting file to be included in a Faspex package.  This article steps you through the process of modifying your Faspex On Demand 3.+ to support this option.

 

Prerequisites

This article assumes the following:

  • You have purchased and booted up Faspex On Demand 3.+ (FOD)
  • You have created an S3 bucket
  • You know your S3 Access ID and Secret Key
  • You know how to SSH as root to your Faspex On Demand Instance

 

Choose a Configuration Option

If you are already using On Demand Faspex, there are extra steps involved to create a second file server & file source. Pick Option A or B as appropriate:

  • Choose Option A if you are new to On Demand Faspex
  • Choose Option B if you are already using On Demand Faspex with a non-S3 file source

 

 

Configuration option A: use existing Server and File Source

With this option we will use the existing "my_server" and "sample_source" for S3 storage. 

  1. Login to the instance console as root using SSH
  2. Use vi to edit the file /opt/aspera/etc/aspera.conf
    1. In the stanza "<name>faspex_source</name>", locate the line beginning "<absolute>"
    2. remove "/mnt/ephemeral/source" and replace with your S3 credentials, using this syntax: S3://AccessKeyID:SecretKey@s3.amazonaws.com/BucketName/ as highlighted in the example below. If you are using IAM roles to access S3, then under the <absolute> tab, replace /home/faspex/faspex_packages to  s3://s3.amazonaws.com/BucketName/

    3. Your secret key might contain one or more + or  /  symbols. Replace all + with "%2B" and all  /  with "%2F"

      07c.png
  3. restart the Aspera node daemon with this command:  service asperanoded restart
  4. restart the Aspera Central service with this command:  service asperacentral restart
  5. Log into your Faspex On Demand as the user 'admin' (this is the initial account that is provided with your FOD instance).
  6. Click "New Package", then click the "Source" drop-down menu and choose "sample_source"

    02b.png
  7. Click "Browse for files" and you should see the contents of your S3 bucket.
  8. If you want to change the name "sample_source", you can do so in the Server configuration area. Click the "Server" tab, then the "File Sources" tab, then the "Edit" button for "sample_source". 

    03b.png
  9. Edit the value in the "Name" field and click "Update File Share".
  10. Done

 

 

Configuration option B: add new Server and File Source

If you are already using the existing "my_server" and "sample_source" with non-S3 storage, then you must create a new file server and file source to use with S3 storage. 

  1. Login to the instance console as root using SSH  
  2. Add a new Unix operating system user named "faspex_source2" with the command:  useradd  faspex_source2
  3. Add a new node user named "faspex_source2" with the following command. Use your own instance ID as a password following "-p":  /opt/aspera/bin/asnodeadmin  -a  -u  faspex_source2  -p  your.image.id  -x  faspex_source2
  4. Use vi to edit the file /opt/aspera/etc/aspera.conf
    1. Within the <aaa> stanza, there will be two sections between <user> ... </user> tags, one for "faspex" and one for "faspex_source". We want to duplicate the entire "faspex_source" section so that there are now three <user> ... </user> sections within the <aaa> stanza.

      08b.png
    2. In the new stanza "<name>faspex_source2</name>", locate the line beginning "<absolute>"
    3. remove "/mnt/ephemeral/source" and replace with your S3 credentials, using this syntax: S3://AccessKeyID:SecretKey@s3.amazonaws.com/BucketName/ as highlighted in the example below
    4. Your secret key might contain one or more + or  /  symbols. Replace all + with "%2B" and all  /  with "%2F"

      10c.png
  5. Log into your Faspex On Demand as the user 'admin' (this is the initial account that is provided with your FOD instance).
  6. Click on the Server tab, then on the File Sources sub-tab, then on "Add Server"

    01d.png
  7. Add the following fields, as depicted in the screenshot below. Use "faspex_source2" as the username. The default password is the instance ID.

    02e.png
  8. If clicking "Test Connection" results in "Connection succeeded", then click "Create". You will see a yellow "Added Server..." message at the top
  9. Click on "Add File Source"
  10. Choose a name for your New File Source, such as "S3 bucket". Then click on "Browse"

    03d.png

  11. If this is a new S3 bucket, just click "Select". If you have existing subdirectories in your S3 bucket, browse until you reach the desired location, then click "Select"

    05f.png
  12. Log into your Faspex On Demand as the user 'admin' (this is the initial account that is provided with your FOD instance)
  13. Click "New Package", then click the "Source" drop-down menu and choose "sample_source"

    02b.png
  14. Click "Browse for files" and you should see the contents of your S3 bucket
  15. Done
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk