Run on a cluster or in a container using
The pipeline itself does not specify any container or cluster resources, so you could just roll your own and submit individual jobs to a cluster however you choose. The easier way is to use
looper's built-in template system, which
looper uses to build flexible shell scripts for job submission. These templates can be used to run jobs in a container, to submit to a cluster resource manager, or both.
looper templates, we must create a computing environment configuration file called
PEPENV. In short, you will need to:
- Set up a compute configuration file that includes a containerized or cluster compute template (or both).
- Point the environment variable
PEPENVto the location of this file.
- Run the pipeline with
looper run --compute PACKAGE(where
PACKAGEis specified in your
This enables you to adjust your computing preferences on-the-fly when you run a project.
The complete description of setting up
looper to use
PEPENV is generic to any pipeline, and therefore omitted from this readme. If you want to use looper with containers or clusters, you should consult the complete docs in the pepenv readme. Further instructions can also be found in the documentation on configuring looper to use a cluster and configuring looper to use linux containers.