1. experipy.grammar - Composing experiments

This module provides the core elements which compose the Experipy grammar: Executables, Wrappers, Pipelines, and Groups. These elements facilitate specifying programs to execute as well as the files they depend on.

1.1. Element objects

class experipy.grammar.Element(inputs=None, outputs=None)

The Element class forms the grammar’s base class.

Parameters:
  • inputs (list) – A list of strings which are the names of files that the Element relies on for input. These will be copied to the run directory when an Experiment is used to run the Element.
  • outputs (list) – A list of strings which are the names of files that the Element is expected to generate as output. These will be copied from the run directory when an Experiment is used to run the Element.
inputs()

Generator which yields the Element’s input files

outputs()

Generator which yields the Element’s output files

1.2. Executable objects

The Executable class extends the base Element class by providing an abstraction for describing a program executable. Once instantiated, converting an Executable object to a string will yield the command string that will be entered into the shell script.

class experipy.grammar.Executable(prog, opts=None, wait=True, **kwargs)

Executable objects should represent a single program and its arguments.

Parameters:
  • prog (str) – The name of the program executable.
  • opts (list) – A list of command line options to pass to the program. Defaults to an empty list if not provided.
  • wait (bool) – If False, a ‘&’ will be appended to the argument list, indicating to the shell that it should background the program instead of blocking on it. Defaults to True.

1.3. Wrapper objects

Wrappers are executables which accept another Executable and its arguments as a parameter, and incorporates the wrapped Executable into its argument list and collection of inputs and outputs.

class experipy.grammar.Wrapper(prog, opts, wrapped, **kwargs)

Wrapper objects allow specification of a program which wraps another.

Wrappers are a subclass of Executable which allow specification of programs such as GDB or Valgrind, which wrap around another program to alter or observe its execution.

Parameters:
  • prog (str) – The name of the program executable.
  • opts (list) – A list of command line options to pass to the program. Must minimally contain a string having the value ‘[[wrapped]]’, which indicates where the wrapped executable should be inserted into the wrapping executable’s argument list.
  • wrapped (experipy.Executable) – The wrapped Executable. Inputs and outputs specified to wrapped will be included in the resultant object’s inputs and outputs.
  • wait (bool) – If False, a ‘&’ will be appended to the argument list, indicating to the shell that it should background the program instead of blocking on it. Defaults to True.
inputs()

Generator which yields the Wrapper’s input files

outputs()

Generator which yields the Wrapper’s output files

1.4. Pipeline objects

The Linux shell supports piping of output from one program into the input of another. Pipelines provide a mechanism to support that feature in the generated shell scripts.

class experipy.grammar.Pipeline(*parts, **kwargs)

Pipeline objects allow specification of pipelined workflows.

A Pipeline takes one or more Element parts, and joins them with a ‘|’ operator, indicating to the shell that each part should recieve its input from the previous part, and provide its output to the next.

Parameters:*parts – One or more Executables or Wrappers to be chained together into a pipeline. Inputs and outputs to the individual parts will be included in the Pipeline’s inputs and outputs.
inputs()

Generator which yields the Pipeline’s input files

outputs()

Generator which yields the Pipeline’s output files

1.5. Group objects

Groups allow generation of more complex experiment behavior than the execution of a single Executable, Wrapper, or Pipeline.

class experipy.grammar.Group(*parts, **kwargs)

Group objects allow specification of Executables to be run in order.

In the resultant script, a Group’s parts will be included one after another, in the order they were specified. Groups should be used when specifying complex experiments involving multiple steps like set up or post-processing, or combined with the wait parameter to Executable to specify programs which should be run concurrently. A Group can also be used as a part in another Group.

Parameters:*parts – One or more Elements to be placed into the script. Inputs and outputs to the individual parts will be included in the Group’s inputs and outputs.
inputs()

Generator which yields the Group’s input files

outputs()

Generator which yields the Group’s output files

1.6. Block objects

Blocks are simple text blocks that will be rendered into the runscript without additional processing.

class experipy.grammar.Block(text, **kwargs)

Blocks allow arbitrary text to be rendered into the script without further processing or enforcing other grammatical rules.

Parameters:text (str) – The text to be rendered.