Scheduling Scripts on Linux/Mac (Cron Jobs)

  flowchart LR
  A["works in your terminal"] --> B["cron runs it"]
  B --> C{"fails?"}
  C --> D["PATH is minimal -- 'python' may not resolve"]
  C --> E["cwd is HOME, not the project -- relative paths break"]
  C --> F["no virtualenv activated -- imports fail"]
  D --> G["use an absolute interpreter path: /path/venv/bin/python"]
  E --> H["cd in the command, or use absolute paths everywhere"]
  F --> G
  G --> I["redirect output to a log -- cron mail is easy to miss"]
  H --> I

Cron basics

Cron runs commands on a schedule.

Common gotchas:

• minimal PATH
• different working directory

Pattern

• use absolute paths
• redirect outputs

Example crontab entry:

0 9 * * * /usr/bin/python3 /home/user/job.py >> /home/user/job.log 2>&1

Tip

Use logging in the script, not only stdout.

Reading a cron line

A crontab entry has five time fields, then the command.

┌───────── minute (0-59)
│ ┌─────── hour (0-23)
│ │ ┌───── day of month (1-31)
│ │ │ ┌─── month (1-12)
│ │ │ │ ┌─ day of week (0-7, Sunday is 0 or 7)
│ │ │ │ │
0 9 * * * command

A * means every value. Some examples:

Edit your table with crontab -e, list it with crontab -l. Each user has a separate table, and the time zone is the system's. Test a schedule on a site such as crontab.guru before you rely on it.

The environment problem

The diagram above shows the main trap. Cron does not run your shell startup files, so PATH is short (often just /usr/bin:/bin), the working directory is your home folder, and no virtual environment is active. A script that works in your terminal can fail silently under cron.

The fixes are all about being explicit:

• Use the absolute path to the interpreter, ideally the one inside your virtual environment: /home/user/project/.venv/bin/python.
• Use absolute paths for the script, input files, and log file. Or cd first.
• Set variables you need in the crontab, or load a .env file in the script.
• Redirect output to a file. >> job.log 2>&1 appends both standard output and errors.

A robust setup

Put the command in a small wrapper script so the crontab line stays simple and you can run the wrapper by hand to test it.

#!/usr/bin/env bash
# /home/user/project/run_job.sh
set -euo pipefail
cd /home/user/project
exec /home/user/project/.venv/bin/python job.py --source /home/user/data >> /home/user/project/job.log 2>&1

Make it executable with chmod +x run_job.sh. The crontab entry is then:

0 9 * * * /home/user/project/run_job.sh

set -euo pipefail makes the shell stop on the first error instead of continuing. If you run the wrapper by hand and it works, and cron runs the same file, then most environment surprises are gone.

Overlapping runs

If a job takes longer than the interval, cron starts a second copy while the first is still running, and they may fight over the same files. Use flock to allow one at a time: */5 * * * * flock -n /tmp/job.lock /home/user/project/run_job.sh. With -n a second copy exits immediately when the lock is held.

macOS

Cron still works on macOS, but the system may need to be allowed Full Disk Access for cron to read some folders. The native tool is launchd. On servers that use systemd, timers are a stronger alternative, with logging through journalctl.

Common mistakes

• Relative paths and a bare python.
• A % in the command. In a crontab, % means a newline, so escape it as \%, which matters for date +%Y.
• No output redirect, so failures vanish or go to local mail nobody reads.
• Forgetting the time zone, or that the machine was asleep or off at the scheduled time. Cron does not catch up on missed runs.
• Editing the file directly instead of with crontab -e.

🧪 Try It Yourself

Exercise 1 – List Files with os.listdir

Press Run to see what your code prints.

Exercise 2 – Join Paths with os.path.join

Press Run to see what your code prints.

Exercise 3 – Write and Read a File

Press Run to see what your code prints.