Getting your parameters: aenv

The parameters for tasks can be stored in an CSV file, where the first row is simply the names of the parameters, and each consecutive row represents the values of these parameters for a specific experiment, i.e., computational task.

aenv will use the task identifier as an index into this CSV file, and define environment variables with the appropriate values for that task. As an example, consider the following PBS script:

#!/bin/bash
...
alpha=0.5
beta=-1.3
Rscript bootstrap.R $alpha $beta
...

However, this approach would lead to as many job scripts are there are parameter instances, which is inconvenient to say the least.

This computation would have to be done for many values for alpha and beta. These values can be represented in an CSV file, data.csv:

alpha,beta
0.5,-1.3
0.5,-1.5
0.5,-1.9
0.6,-1.3
...

The job script can be modified to automatically define the appropriate values for alpha and beta specific for the task.

#!/bin/bash
...
source <(aenv --data data.csv)
Rscript bootstrap.R $alpha $beta
...

aenv will use the value of the task identifier to read the corresponding row in the data.csv CSV file, and export the variables alpha and beta with those values.

aenv has a number of options that can be useful in some situations.

  1. --sniff <nr_bytes>: the number of bytes used to determine the CSV dialect. It may be useful to increase this from the default value of 1024 bytes for files with many columns, or large line lengths.
  2. --no_sniffer : for data files with a single column, the sniffer gets confused, so it should be switched off.
  3. --id <id>: used to specify your own line number in the CSV file to determine the variables' values. This can be used in conjunction with a queue system or scheduler that atools doesn't currently support. This defaults to the task identifier as set by the queue system or scheduler.
  4. --shell bash|sh|tcsh|csh: shell syntax generated by aenv, defaults to bash/sh.
  5. --conf <config-file>: use the specified configuration file, rather than the default one.

Help on the command is printed using the --help flag.