Process Engines

Once you have defined a workflow, there are different ways to run the processes it consists of. The simplest way is to turn the workflow into a Guile script that sets up the desired environment and then executes the workflow processes on the current machine. This is what the simple-engine does.

The drmaa-engine submits generated process scripts to an HPC cluster scheduler implementing DRMAA version 1, such as the various incarnations of Grid Engine or Slurm. To use this engine you must first set the environment variable GUILE_DRMAA_LIBRARY to the location of the libdrmaa.so shared library provided by your HPC scheduler. Here is an example command from a system using Altair Grid Engine:

export GUILE_DRMAA_LIBRARY=/opt/age-8.7.0/drmaa/lib/lx-amd64/libdrmaa.so

The grid-engine is similar to the simple-engine in that it generates a shell script, with the difference that it also includes resource variable definitions for submission to a Grid Engine scheduling system. The resource variables are derived from the process run-time field. This process engine is deprecated in favor of drmaa-engine.