2. experipy.exp - The Experiment Runner¶

This module provides the Experiment class for running compositions in the grammar, as well as the Exp Namespace for controlling and configuring Experiment behavior.

2.1. An Example¶

from experipy.exp       import Experiment
from experipy.grammar   import Executable

exp = Experiment(Executable("echo", ["Hello World"]),
                 expname="test",
                 destdir="results")
exp.run()

This will run the program echo with the argument Hello World in a directory in /tmp, writing the output and error, along with timing information, to the directory results. Directories will be created as needed.

2.2. Experiment objects¶

class experipy.exp.Experiment(cmd, expname='exp', destdir=None)¶

Experiment objects perform the generation and execution of runscripts.

Once a composition has been specified in the grammar, wrapping it in an Experiment allows the user to generate a shell script as a string using the make_runscript method. The run and queue methods provide mechanisms for executing the generated scripts.

Parameters:

cmd (experipy.Element) – A composition of experipy Elements such as Executable and Group, which defines the behavior the user wishes the Experiment to perform.
expname (str) – A name to be used for identifying the experiment. Defaults to Exp.defname, which defaults to “exp”.
destdir (str) – An optional path to a directory where the results from running the experiment should be stored. If None, expname will be used.

make_runscript(preamble='#!/bin/bash', rm_rundir=True)¶

Create a string containing the experiment rendered as a shell script.

Parameters:	preamble (str) – The first line(s) of the runscript. Defaults to Exp.shebang, which defaults to “#!/bin/bash”. rm_rundir (bool) – If True, a line deleting the experiment’s working directory will be added to the end of the script. Defaults to True.
Returns:	A run script as described by the composition provided to the Experiment.
Return type:	str

queue(h=False, n=False, q=None, A=None, **kwargs)¶

Submit the experiment to a job queuing system as a PBS script.

Generates a script with a PBS script header, writes the script to the results directory, and then submits it to the job queuing system by running the command qsub as a subprocess.

Parameters:	h (bool) – Will add a `-h` to pbs headers if True, Default is False. n (bool) – Will add a `-n` to pbs headers if True, Default is False. q (str) – Optionally request a resource queue. A (str) – Optionally name the account to charge for this job. **kwargs – The remaining keyword arguments will be combined into resource requests with -l.

run(rm_rundir=True)¶

Execute the experiment as a subprocess of the current process.

Generates a run script, writes that script to the results directory, and then executes the script as a subprocess of the current process. The time the script takes to execute, including setup and clean up time, is recorded. This function blocks until the experiment is complete.

Parameters:	rm_rundir (bool) – If True, the directory created for running the experiment will be deleted at the end of the experiment. Defaults to True.

sbatch(**kwargs)¶

Submit the experiment to a Slurm cluster as an sbatch script.

Generates a script with a Slurm script header, writes the script to the results directory, and then submits it to the job queuing system by running the command sbatch as a subprocess.

Parameters:	**kwargs – Keyword arguments will be translated to SBATCH directives of the form `#SBATCH --<key>=<value>`. Underscores in keyword argument names will be substituted for dashes in the emitted SBATCH directives. For example, `cpus_per_task=4` will be translated to `#SBATCH --cpus-per-task=4`.

2.3. The Exp Namespace¶

Default values for paths and filenames in the Experiment class are controlled by a Namespace called Exp. These defaults are listed below, and can be overridden by setting a new value in the .experipyrc under the [Exp] section.

Key	Default Value	Description
shebang	#!/bin/bash	The first line of the generated shell scripts.
rundir	/tmp	Path to the directory where the experiment is going to be run.
defname	exp	Default name of experiments.
runsh	run.sh	Name of the generated shell scripts.
out	raw.out	Name of the file which will collect the experiment’s standard output.
err	raw.err	Name of the file which will collect the experiment’s standard error.
timing	harness_time.out	When an experiment is run using `run()`, its run time will be captured in this file.