Skip to content

Introduction to Batch Jobs

You may also be interested in:

What are Batch Jobs?

Batch jobs differ from interactive jobs and graphical jobs because they do not require user input while running. Instead, the user writes a script containing the instructions (code) that is sent to a compute node via the scheduler (Slurm). This allows your workflow to run automatically without you needing to be physically present. Here are a few benefits of using batch jobs:

  1. No Need to Stay Logged In: You don’t have to remain logged into the HPC system for your work to continue. This avoids potential issues like your terminal timing out, local internet disruptions, or your computer going to sleep—all of which could prematurely end your analysis, especially for long-running jobs.

  2. Submit Many Jobs at Once: Some workflows require running hundreds or thousands of analyses. For example, you might want to run the same script with different input values multiple times. Doing this interactively could be cumbersome or even impossible. Batch jobs can easily handle this use case.

Batch Job Workflow and Analogy

Think of a batch job like a researcher who wants something custom-made at a factory. There are a few steps they need to take:

  1. Get the Address: First, they need to know where the factory is so they can contact the right person to make their request.
  2. Provide Instructions: Next, they need to write instructions, or schematics, for the person who will do the manufacturing.
  3. Send the Instructions: Finally, they need to send these instructions to the factory so the builder can receive them and start working.

post office and factory

We'll continue with this analogy, breaking down each step in more detail in the Batch Tutorial.