Faspex on Demand with RDS, Autoscale and Elastic IP

IN THIS ARTICLE:

 

Overview

This article describes a process for provisioning your Faspex on Demand within the Amazon Autoscale service. This deployment is relevant for organizations that want to add some robustness to their Faspex on Demand deployment. The AWS Autoscale service is capable of monitoring a single instance, and if that instance should be deleted, a new instance will be launched. Here is a diagram of the solution.

 

FOD_with_Autoscale.png

 

Note: This article covers a single instance of Faspex running in Autoscale. For multiple instances, additional software and configuration is required to manage the services that cannot run on all systems concurrently.

 

Prerequisites

  1. You have a subscription to Aspera Faspex on Demand (FOD).
  2. You are familiar with the AWS services:  S3, Autoscale, RDS, Route53, Elastic IP, IAM role
  3. You have obtained an Elastic IP (EIP) for your Faspex on Demand instance
  4. You can ssh to your Faspex on Demand instance, using the ssh key
  5. You have created an IAM role for access to S3
  6. You have created a security group for your Faspex on Demand instance
  7. You have an Access Key and Secrete for use with your instance
  8. (Optional - but recommended) You have a hostname for your Faspex on Demand that is registered in DNS pointing to your EIP
  9. (Optional - but recommended) You have a SSL certificate for your Faspex on Demand hostname.

 

Procedure - Overview

  1. Configure your Faspex to use your EIP, Hostname and SSL certificate
  2. Configure your Faspex on Demand to use RDS
  3. Install and configure an EIP auto-configuration utility
  4. Create a new AMI image of your configured system
  5. Configure a new Autoscale group and launch configuration for this new image

 

Procedure - details

  1. Configure Faspex EIP, Hostname and SSL Certificate
    1. EIPs are assigned using the AWS console, or command line.  We urge you to assign the EIP address early in the FOD boot process.  The FOD boot scripts are designed to detect the EIP and auto configure FOD accordingly.  If you configure your EIP after FOD is fully booted, you will have to manually adjust a configuration file.  That process is documented in this knowledge base article.
    2. Hostname:  To assign a hostname to your FOD instance, use the asctl apache:hostname host utility.  This is available for root users, from the FOD command line and documented in the FOD admin guide.  We strongly urge you to use a proper hostname, that is associated with an SSL certificate (see step below).  
    3. Hostname:  Configure your Aspera server to have knowledge of the Faspex hostname.  This can be done via the aspera.conf file.  Here is an example command where I set the hostname to faspex.example.com
      asconfigurator -x "set_server_data;server_name,faspex.example.com"
      You can confirm that this worked by looking at the file /opt/aspera/etc/aspera.conf. You will see the configuration in the <server_name> section.
    4. Configure the Faspex server to stop re-configuring hostname on each reboot.  This is accomplished by editing the init script "/etc/init.d/asp-ondemand-secondboot-reconfigure, and commenting out the following line (e.g. add the '#' at the beginning of the line) (NOTE: The default behavior of the system is designed for dynamic environments, where the IP address may chance on system reboot, which obviously will not work in this fixed IP / Hostname configuration.)
      #  /opt/aspera/ondemand/bin/reconfig-fod.sh -A
    5. SSL Certificate: Installing your proper SSL certificate can be as easy as replacing the existing self signed SSL certificates.  This is covered in this KB article:  How to install SSL Certificates on your FOD.
  1. Configure Faspex on Demand for RDS
    1. RDS is the AWS database service.  Setup and configuration of FOD for RDS is covered in this KB article:  How to configure FOD for RDS.
  2. Install and configure an EIP auto-configuration utility
    1. Upon system reboot, it is required that FOD be auto assigned the same EIP.  As of the time of this article, there is no AWS facility for auto-assigning an EIP from within a booting instance.  (The AWS tooling assumes there is a human booting the system, or there is a pool of addresses). Fortunately someone has solved this problem and posted the solution. The steps are documented below.
    2. Login via SSH to your FOD system and sudo to root.  Install the aws-ec2-assign-elastic-ip utility:
      # curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
      # python get-pip.py
      # pip install aws-ec2-assign-elastic-ip
    3. Test the utility with your EIP, Access key and Secret, for example:
      aws-ec2-assign-elastic-ip --region us-west1 --secret-key FQkJs8uwSYijGcy4/IawBr0FsRCr8tPb3nDkCoFe --access-key ALIWIUKNOGS43OELHMTA --valid-ips 54.221.36.9
  3. Configure the default user data init script to run earlier in the boot process.  The default init script is configured to be the last init script run in run level 3.  In our case, we want the EIP address assigned earlier in the boot sequence.  You need to adjust the default symbolic link, as follows.
    cd /etc/rc.d/rc3.d/
    rm -rf S99cloud-init-user-scripts
    ln -s ../init.d/cloud-init-user-scripts S25cloud-init-user-scripts
  4. Create a new AMI image from your configured Faspex on Demand system
    1. Prior to creating a new image, you should shut down all services and clear log files
    2. Login to your AWS console, navigate to the EC2 > Images, select your running FOD image and from the ACTIONS menu, select Image > Create Image.
    3. Make a note of the new image AMI ID
  5. Configure a launch configuration and autoscale group
    1. Login to the AWS console, and navigate to the EC2 > Autoscale > Launch configuration.
    2. Click on Create new Launch configuration.
    3. Navigate to My AMI's and select the newly created AMI, click next.
    4. Select the Instance type (e.g. m3-xlarge) and click next.
    5. Specify the configuration details (e.g. IAM Role and Name) and click next.
    6. Select Advanced and add the following script into the userdata section. Adjust the script to your EIP, Access Key, Secret and region:
      aws-ec2-assign-elastic-ip --region us-west1 --secret-key FQkJs8uwSYijGcy4/IawBr0FsRCr8tPb3nDkCoFe --access-key ALIWIUKNOGS43OELHMTA --valid-ips 54.221.36.9
    7. Confirm that the storage space meets your requirement and click next.
    8. Specify the security group that provides sufficient access (e.g. HTTP(s) and TCP/UDP 33001)
    9. Review the configuration options and select an ssh key required to access the system.
    10. At this point, you have a launch configuration and you need to create the Autoscaling group.
    11. Create a new Autoscaling group, specify a name for your group.  
    12. Leave the default of 1 instance for this group (note;  For multiple Faspex instances - there are additional steps).
    13. (Optional) Specify the subnet that you are running your instance in.
    14. Specify an availability zone for your instance
    15. (Optional) Under advanced option, specify a custom health check grace period.
    16. (Optional) Create an autoscaling group notification and click next.
    17. (Optional - but suggested) Create a tag for your auto scaling group.
    18. Review all configured options and save the auto scaling group.

At this point, the AWS Autoscale group should launch an instance according to your specification.  You can view the running instance in your EC2 instances.  If nothing is there, please navigate back to the Autoscale Group configuration and review the Activity History and Instances for more information.

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk