Shares On Demand with RDS, Autoscale, and Elastic IP

IN THIS ARTICLE:

 

Overview

This article describes a process for provisioning your Shares on Demand instance within the Amazon Autoscale service. This deployment is relevant for organizations that want to add some robustness to their Shares on Demand deployment. The AWS Autoscale service is capable of monitoring multiple instances, and should any one instance be deleted, a new instance will be launched.

 

Note: Aspera Console will be disabled in these steps.

 

Prerequisites

  1. You have a subscription to Aspera Shares on Demand (SHOD).
  2. You are familiar with the AWS services:  S3, Autoscale, RDS, Route53, Elastic IP, IAM roles.
  3. You have obtained an Elastic IP (EIP) for your Shares on Demand instance.
  4. You can ssh to your Shares on Demand instance, using the ssh key.
  5. You have created an IAM role for access to S3.
  6. You have created a security group for your Shares on Demand instance.
  7. You have an Access Key and Secret Access Key for use with your instance.
  8. (Optional - but recommended) You have a hostname for your Shares on Demand that is registered in DNS pointing to your EIP.
  9. (Optional - but recommended) You have an SSL certificate for your Shares on Demand hostname.

 

Procedure - Overview

  1. Configure your Shares to use your EIP, Hostname, and SSL certificate.
  2. Configure your Shares on Demand to use RDS.
  3. Install and configure an EIP auto-configuration utility.
  4. Create a new AMI image of your configured system.
  5. Configure a new Autoscale group and launch configuration for this new image.

 

Procedure - details

  1. Configure Shares EIP, Hostname and SSL Certificate:
    1. EIPs are assigned using the AWS console, or command line.  We urge you to assign the EIP address early in the SHOD boot process.  The SHOD boot scripts are designed to detect the EIP and auto configure SHOD accordingly.  If you configure your EIP after SHOD is fully booted, you will have to manually adjust a configuration file.  That process is documented in this knowledge base article.
    2. Hostname:  Configure your Aspera server to have knowledge of the SHOD hostname.  This can be done via the aspera.conf file.  Here is an example command where I set the hostname to shod.example.com
    3. asconfigurator -x "set_server_data;server_name,shod.example.com"
    4. You can confirm that this worked by looking at the file /opt/aspera/etc/aspera.conf.  You will see the configuration in the <server_name> section.
    5. Configure the Shares server to stop re-configuring hostname on each reboot.  This is accomplished by editing the init script "/etc/init.d/asp-ondemand-secondboot-reconfigure, and commenting out the following line (e.g. add the '#' at the beginning of the line) (NOTE: The default behavior of the system is designed for dynamic environments, where the IP address may change on system reboot, which obviously will not work in this fixed IP / Hostname configuration.)
  2. #  /opt/aspera/ondemand/bin/reconfig-shod.sh -A
    1. SSL Certificate: Installing your proper SSL certificate can be as easy as replacing the existing self signed SSL certificates.  This is covered in this KB article:  How to install SSL Certificates on your SHOD.
    2. Restart noded:
    3. service asperanoded restart
  1. Configure Shares on Demand for RDS
    1. RDS is the AWS database service.  Setup and configuration of SHOD for RDS is covered in this KB article:  How to configure SHOD for RDS.
  2. Install and configure an EIP auto-configuration utility. (NOTE: Skip this step if you will be using an ELB or multiple Shares instances withing your Autoscaling group).
    1. Upon system reboot, it is required that SHOD be auto assigned the same EIP.  As of the time of this article, there is no AWS facility for auto-assigning an EIP from within a booting instance.  (The AWS tooling assumes there is a human booting the system, or there is a pool of addresses). Fortunately someone has solved this problem and posted the solution. The steps are documented below.
    2. Login via SSH to your SHOD system and sudo to root.  Install the aws-ec2-assign-elastic-ip utility:
      # curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
      # python get-pip.py
      # pip install aws-ec2-assign-elastic-ip
    3. Test the utility with your EIP, Access key and Secret key, for example:
      aws-ec2-assign-elastic-ip --region us-west1 --secret-key FQkJs8uwSYijGcy4/IawBr0FsRCr8tPb3nDkCoFe --access-key ALIWIUKNOGS43OELHMTA --valid-ips 54.221.36.9
  3. Configure the default user data init script to run earlier in the boot process. The default init script is configured to be the last init script run in run level 3. In our case, we want the EIP address assigned earlier in the boot sequence. You need to adjust the default symbolic link, as follows.
  4. cd /etc/rc.d/rc3.d/
  5. rm -rf S99cloud-init-user-scripts
  6. ln -s ../init.d/cloud-init-user-scripts S25cloud-init-user-scripts
  7. Create a new AMI image from your configured Shares on Demand system
    1. Prior to creating a new image, you should shut down all services and clear log files
    2. Login to your AWS console, navigate to the EC2 > Images, select your running SHOD image and from the ACTIONS menu, select Image > Create Image.
    3. Make a note of the new image's AMI ID.
  8. Configure a launch configuration and autoscale group
    1. Login to the AWS console, and navigate to the EC2 > Autoscale > Launch configuration.
    2. Click on Create new Launch configuration.
    3. Navigate to My AMI's and select the newly created AMI, click next.
    4. Select the Instance type (e.g. m3-xlarge) and click next.
    5. Specify the configuration details (e.g. IAM Role and Name) and click next.
    6. Select Advanced and add the following script into the userdata section. Adjust the script to your EIP, Access Key, Secret and region (NOTE: Skip this step if you will be using an ELB or multiple Shares instances withing your Autoscaling group).:
      #!/bin/bash aws-ec2-assign-elastic-ip --region us-west1 --secret-key FQkJs8uwSYijGcy4/IawBr0FsRCr8tPb3nDkCoFe --access-key ALIWIUKNOGS43OELHMTA --valid-ips 54.221.36.9
    7. Confirm that the storage space meets your requirement and click next.
    8. Specify the security group that provides sufficient access (e.g. HTTP(s) and TCP/UDP 33001)
    9. Review the configuration options and select an ssh key required to access the system.
    10. At this point, you have a launch configuration and you need to create the Autoscaling group.
    11. Create a new Autoscaling group, specify a name for your group.  
    12. Select the number of instances to run.
    13. (Optional) Specify the subnet that you are running your instance in.
    14. Specify an availability zone for your instance
    15. (Optional) Under advanced option, specify a custom health check grace period.
    16. (Optional) Create an autoscaling group notification and click next.
    17. (Optional - but suggested) Create a tag for your auto scaling group.
    18. Review all configured options and save the auto scaling group.

At this point, the AWS Autoscale group should launch an instance according to your specification.  You can view the running instance in your EC2 instances.  If nothing is there, please navigate back to the Autoscale Group configuration and review the Activity History and Instances for more information.

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk