Using asconfigurator

The asconfigurator:

Instead of manually editing the aspera.conf or implementing their own XML parsers, most Aspera tools use asconfigurator. Once the basics of asconfigurator are understood, it is easy to quickly and safely edit the aspera.conf with one or two commands. asconfigurator parses, validates, and writes well formed XML properly nested and to spec.

syntax:

There are multiple ways to run asconfigurator, but the basic two modes are from STDIN and from a command string passed in as an argument. For the purpose of this document, asconfigurator commands will be passed in from the CLI rathen than STDIN.

  1. This mode of asconfigurator takes in a command and edits or reads from the default aspera.conf (typically /opt/aspera/etc/aspera.conf on Linux). The -F means to print out to STDOUT the response from the command. A -f is similar, but it supresses the the response to STDOUT

    asconfigurator -F "cmd_string"
  2. In the second form, the additional argument is a new or alternate configuration file. If the file does not exist, it creates a fresh configuration, although it should be noted, this only works for set_node_data, set_server_data and other non-filtered commands (discussed below). When tried with set_user_data commands a Segmentation Fault occurs. If the file exists, it assumes it is another aspera.conf

    asconfigurator -F "cmd_string" config.conf
  3. In the third and last form, the two file arguments pass in a template and new aspera.conf. With this instantiation, an already existing aspera.conf can be used as the basis for a new one.

    asconfigurator -F "cmd_string" base.conf new.conf

In all 3 instances above asconfigurator takes in a “cmd_string” argument. The “cmd_string” takes two basic forms.

The last note on syntax:  In addition to the -F and -f flags, there is a -x.  This option is like -f in terms of verbosity, but it treats the command string differently.  With -F and -f the commands replace every option in the section specified with the new options passed in the command string.  The -x option will just change the option specified and it will leave the other setting unchanged.  In this way, it acts like a "modify" operator.

Commands for Getting and Setting

Since the goal of the tool is to safely parse the aspera.conf, the asconfigurator makes a great tool for not only setting, but also getting data. In addition to stating what is set by explicit values in the configuration, the asconfigurator will also note default values that apply in liu a set value.

One of the easiest ways to find out a good portion of the command for setting (and getting), is to review the aspera.conf spec. This is done using asuserdata (which is not symlinked).

# /opt/aspera/bin/asuserdata -s
<conf version="2">
...
<default>
...
    <authorization>
        <transfer>
            <in>
                <value>allow</value> <!-- Incoming Transfers: trimmed_char_ptr -->
                    <!-- asconfigurator -f "set_node_data;authorization_transfer_in_value,<value>" -->
                    <!-- Choices: allow, deny, token -->
                    <!-- The default setting of allow allows users to transfer to this computer.
                         Setting this to deny will prevent transfers to this computer.
                         When set to token, only transfers initiated with valid tokens will be allowed to transfer to this computer.
                         Token-based transfers are typically employed by web applications such as Faspex and require a Token Encryption Key. -->
...

This method not only documents the commands for asconfigurator, but it also provides information on the proper values allowed for each parameter.

Sadly, this does not cover everything. For the user and group level commands, the spec mostly glosses over them. Luckily, there are ways to figure out the remaining commands and parameters. Working back from previous examples in the spec, it is easy to see a pattern to asconfigurator commands. The following strings command can provide the major commands:

strings /opt/aspera/bin/asconfigurator | egrep "[sg]et_.+_data"

The values returned when I ran this (on an early 2.8 Enterprise Server) show the two distinct command strings:

Getting info

  • get_user_data - Getting info at the User level. Requires additional arguments to specify a particular user.
  • get_group_data - Getting info at the Group level. Requires additional arguments to specify a particular group.
  • get_node_data - Get “global” info on the node, such as default docroot and RTT autocorrect.
  • get_trunk_data - Get VLINK/trunk info.
  • get_central_server_data - Get info on the Central server (Web Services/SOAP) settings.
  • get_database_data - Get info on the database logging info.
  • get_server_data - Get info on the server info used by the Node API.
  • get_http_server_data - Get info on HTTP Fallback server.
  • get_default_data - Quickly get all the default info.

Setting info

  • set_user_data - Set info at the User level. Requires additional arguments to specify a particular user.
  • set_group_data - Set info at the Group level. Requires additional arguments to specify a particular group.
  • set_node_data - Set “global” info on the node, such as default docroot and RTT autocorrect.
  • set_trunk_data -Set VLINK/trunk info.
  • set_central_server_data - Set info on the Central server (Web Services/SOAP) settings.
  • set_database_data - Set info on the database logging info.
  • set_server_data - Set info on the server info used by the Node API.
  • set_http_server_data - Set info on HTTP Fallback server.

A Note on Deleting

A third type of command is possible -- deleting.  In the case where a user, group, or VLink needs to be removed from the configuration  These operators require a filter (described later) just like the get_/set_ methods for users, groups, and VLinks.:

  • delete_user - Delete the user specified.
  • delete_group - Delete the group specified.
  • delete_trunk - Delete the VLink specified.

The Command String

The command string is a simple semi-colon delimited list of arguments. The first argument is one of the principal statements above. In the case of most get_ statements, they do not require any other arguments. A single command string does not need a semi-colon at the end. For example, the best way to see the node settings is via the following:

asconfigurator -F "get_node_data"

In the case of a get_ , set_, or delete_ that requires a filtering argument (for user, group, trunk, etc), the next argument is second. Knowing what the filtering argument is can be relatively simple to determine. The best way is to run asconfigurator with the  get_ command on the type of data desired, and the output will note what is missing. For example:

# asconfigurator -F "get_user_data"
failure
"protocol_version","1.1"
error,4,"Missing Data","Parameter not found. Expecting user_name"
end_of_reply 

This indicates that the user_name is required. So, a follow-up get_ for the user “faspex" is done like so:

# asconfigurator -F "get_user_data;user_name,faspex"
success
"protocol_version","1.1"
"user","faspex","user_value","AS_NULL","AS_NULL","AS_NULL","AS_NULL","0"
"user","faspex","name","AS_NULL","AS_NULL","AS_NULL","AS_NULL","AS_NULL"
"user","faspex","group","AS_NULL","AS_NULL","AS_NULL","AS_NULL","AS_NULL"
...

The third argument onward (or second argument if no filtering argument needed) on a set_ is used to assign a values to the specific portion of the configuration.

Finding a Parameter to Change

Outside of the spec from  asuserdata -s , the best way to find out what values are chageable or settable is to use the equivalent get_ for the set_ that is to be done. The output can be verbose, so a pipe to grep is normally done to limit the output to a specific set of results. For example, knowing the string needed for the RTT Autocorrect can be found like so:

# asconfigurator -F "get_node_data" | grep -i rtt
"node","transfer_protocol_options_rtt_autocorrect","true","false"
"node","transfer_protocol_options_rtt_histlen","AS_NULL","0"
"node","transfer_protocol_options_rtt_variation_index","AS_NULL","0"
"node","transfer_protocol_options_rtt_estimate_skew","AS_NULL","false"
"node","transfer_protocol_options_rtt_reverse_infer","AS_NULL","true"
"node","transfer_protocol_options_rtt_show_dist_on_congestion","AS_NULL","true"

A similar example for finding rate settings for the user “faspex”:

# asconfigurator -F "get_user_data;user_name,faspex" | grep -i rate
"user","faspex","transfer_in_bandwidth_flow_target_rate_cap","AS_NULL","AS_NULL","AS_NULL","AS_NULL","Unlimited"
"user","faspex","transfer_in_bandwidth_flow_target_rate_default","AS_NULL","AS_NULL","AS_NULL","30000","10000"
"user","faspex","transfer_in_bandwidth_flow_target_rate_lock","AS_NULL","AS_NULL","AS_NULL","AS_NULL","false"
"user","faspex","transfer_in_bandwidth_flow_min_rate_cap","AS_NULL","AS_NULL","AS_NULL","AS_NULL","Unlimited"
"user","faspex","transfer_in_bandwidth_flow_min_rate_default","AS_NULL","AS_NULL","AS_NULL","AS_NULL","0"
"user","faspex","transfer_in_bandwidth_flow_min_rate_lock","AS_NULL","AS_NULL","AS_NULL","AS_NULL","false"
"user","faspex","transfer_out_bandwidth_flow_target_rate_cap","AS_NULL","AS_NULL","AS_NULL","AS_NULL","Unlimited"
"user","faspex","transfer_out_bandwidth_flow_target_rate_default","AS_NULL","AS_NULL","AS_NULL","30000","10000"
"user","faspex","transfer_out_bandwidth_flow_target_rate_lock","AS_NULL","AS_NULL","AS_NULL","AS_NULL","false"
"user","faspex","transfer_out_bandwidth_flow_min_rate_cap","AS_NULL","AS_NULL","AS_NULL","AS_NULL","Unlimited"
"user","faspex","transfer_out_bandwidth_flow_min_rate_default","AS_NULL","AS_NULL","AS_NULL","AS_NULL","0"
"user","faspex","transfer_out_bandwidth_flow_min_rate_lock","AS_NULL","AS_NULL","AS_NULL","AS_NULL","false"
"user","faspex","transfer_protocol_options_rate_instru_type","AS_NULL","AS_NULL","AS_NULL","AS_NULL","0"

The output shows several possible options to set to limit, lock, and default in and out bandwidth.

Understanding the get_ Output

The output can be confusing, but it is structured. It can be viewed as columnar data that is comma separated. The number of columns can change depending on the type of value, but in general:

Columns
  1. The type of data being presented, like “user”, “node”, “trunk”
  2. The filtered result – a targetted user, group, or trunk – if the result is a filtered set.
  3. AKA, Column 2 on non-filtered data sets, this is the actual parameter.
  4. Column 4 (or 3) is the set value. If not set, it is AS_NULL
  5. The remaining colums leading to the last are the inheritence chain of values. Typically the are also AS_NULL unless a value set at a higher level (like at the node level for a user)
  6. The last column is the default value used if nothing given.

It is common to see a lot of AS_NULL values. These mean the column is NULL.

For the purpose of this document, it is not necessary to know every column. The important things to note are the default and set values.

For example, review the following:

# asconfigurator -F "get_user_data;user_name,xfer"
...
"user","xfer","transfer_out_bandwidth_flow_target_rate_default","AS_NULL","AS_NULL","AS_NULL","30000","10000"
...

This shows the default out rate for the user “xfer”. The results show that the default for this parameter is 10Mb/s, but the value has been inherited and is set to 30Mb/s.

To show a directly set value, here is the same thing at the node level:

# asconfigurator -F "get_node_data" | grep transfer_out_
...
"node","transfer_out_bandwidth_flow_target_rate_default","30000","10000"
...

A Note on Docroots

Docroots are presented weirdly and have changed in output a couple times. Because of this, they are treated a bit differently. For example:

# asconfigurator -F "get_user_data;user_name,xfer" | grep docroot
"user","xfer","docroot","Docroot #1","absolute","/","AS_NULL","AS_NULL","AS_NULL",""
"user","xfer","docroot","Docroot #1","show_as","AS_NULL","AS_NULL","AS_NULL","AS_NULL",""
"user","xfer","docroot","Docroot #1","description","AS_NULL","AS_NULL","AS_NULL","AS_NULL","AS_NULL"
"user","xfer","docroot","Docroot #1","filter","AS_NULL","AS_NULL","AS_NULL","AS_NULL","*"
"user","xfer","docroot","Docroot #1","filter_type","AS_NULL","AS_NULL","AS_NULL","AS_NULL","Wildcard"
"user","xfer","docroot","Docroot #1","filter_scope","AS_NULL","AS_NULL","AS_NULL","AS_NULL","String"
"user","xfer","docroot","Docroot #1","read_allowed","AS_NULL","AS_NULL","AS_NULL","AS_NULL","true"
"user","xfer","docroot","Docroot #1","write_allowed","AS_NULL","AS_NULL","AS_NULL","AS_NULL","true"
"user","xfer","docroot","Docroot #1","dir_allowed","AS_NULL","AS_NULL","AS_NULL","AS_NULL","true"

Note the “Docroot #1”. At one point the configuration was to allow for multiple docroots, but that future proofing of the configuration is a bit outdated. It is not possible to set multiple docroots. Luckily, the “absolute” parameter is used to set/get a docroot. For example:

# asconfigurator -F "get_user_data;user_name,xfer" | grep absolute
"user","xfer","docroot","Docroot #1","absolute","/","AS_NULL","AS_NULL","AS_NULL",""

This shows the docroot is set to / and different than the default of an empty string. To set this to another value, like /data, use the following:

# asconfigurator -F "set_user_data;user_name,xfer;absolute,/data"

Alternatively, removing a docroot from a user is done via the "docroot_mask" argument:

# asconfigurator -F "set_user_data;user_name,xfer;docroot_mask,*"

Examples of Common (and not Common) Uses

  • Getting node info

    asconfigurator -F "get_node_data"
  • To Set Server name (to “example.com”) and SSH port (back to 22 since default is now 33001) for Node API:

    asconfigurator -F "set_server_data;server_name,exmaple.com;ssh_port,22"
  • To Set RTT Autocorrect (for VMs)

    asconfigurator -F "set_node_data;transfer_protocol_options_rtt_autocorrect,true"
  • To Set Central IP Address (to listen to all IPs)

    asconfigurator -F "set_central_server_data;address,0.0.0.0"
  • To Set Initial Target Rates to one value IN and another OUT on the same instantiation

    asconfigurator -F "set_user_data;user_name,root;transfer_out_bandwidth_flow_target_rate_default,45000;transfer_in_bandwidth_flow_target_rate_default,100000"
  • To Set User Docroot (in this example, for “xfer” user to /data/xfer)

    asconfigurator -F "set_user_data;user_name,xfer;absolute,/data/xfer"
  • To remove a user (in this example, the “xfer” user)

    asconfigurator -F "delete_user;user_name,xfer"
  • To Set Global Docroot (to /)

    asconfigurator -F "set_node_data;absolute,/"
  • To Set S3 Docroot

    asconfigurator -F "set_user_data;user_name,root;absolute,s3://APIUSER:APIKEY@s3.amazonaws.com/"
  • To Set User Token auth (inbound, deny outbound, for user “webint”)

    asconfigurator -F "set_user_data;user_name,webint;authorization_transfer_in_value,token;authorization_transfer_out_value,deny"
  • To Set Token key (for user “webint”)

    asconfigurator -F "set_user_data;user_name,olyzone;token_encryption_key,dghjeygvbqweafygbearf"
  • To Set-up a Trunk (ID 100, capacity 100Mb/s)

    asconfigurator -F "set_trunk_data;id,100;trunk_on,true;trunk_capacity,100000"
  • To change the server_name and keep the other server_data items intact

    asconfigurator -x "set_server_data;server_name,blah.com"
Have more questions? Submit a request

3 Comments

  • Avatar
    Laurent Martin

    A little precision, the command to get the full spec + asconfigurator command is: asuserdata -+

    asuserdat -s only displays the spec, the output in this document is the spec + commands.

  • Avatar
    Laurent Martin

    In order to reset a value to its default value, i.e. remove it from the aspera.conf, set it to the special value : AS_NULL,

    example:

    asconfigurator -F "set_user_data;user_name,olyzone;token_encryption_key,AS_NULL"

  • Avatar
    Laurent Martin

    On Windows (only), asconfigurator will add a line to openssh's "passwd" file : [ES dir]/etc/passwd.

    The line will enable the user in ssh and define the "shell" of the user, as "aspshell.exe" when the user's docroot is not empty, and as switch.exe when the docroot is empty.

    (code for switch.exe is included in Enterprise Server (switch.c), this executes either "cmd.exe" or "sh.exe" depending on how it is called).

    A good practice is to define a user without docroot as "admin user", for use in Console, all configurations are made with "asconfigurator" (including in Console, and ES GUI), and aspshell does not allow execution of asconfigurator.

    When executing asconfigurator, the user needs to have write access to [ES dir]/etc/aspera.conf

     

Please sign in to leave a comment.
Powered by Zendesk