Lesson 5 of 10Diagnostics

12 min · theory and practice · Basics

cron: why a backup did not appear

Check a scheduled job’s timing, user and environment.

You will be able to distinguish a job that never ran from a job that ran and failed.

A script working in your shell does not verify cron. A background job may use a different user, working directory, PATH and handling of special characters. First identify whether the backup is launched by cron or a systemd timer.

Check timing and user

crontab -l shows the current user’s jobs. /etc/crontab and /etc/cron.d include a user field after the five time fields; a user crontab does not. Make sure you are reading the right format.

Commands
crontab -l
cat /etc/cron.d/shop-backup
date
Example output
* * * * * backup /opt/backup/backup.sh >> /var/log/shop-backup.log 2>&1

This is a system job running as backup every minute; the story describes nightly backups, but the exercise shortens the wait.

Reproduce the environment

Check three independent boundaries: finding backup-tool through PATH, resolving config/backup.conf against the working directory, and accessing the destination. A backup must not depend on an administrator’s login shell.

Commands
ls -l /opt/backup/backup.sh
namei -l /opt/backup/config/backup.conf
tail -n 30 /var/log/shop-backup.log

Use the first error in /var/log/shop-backup.log to choose the hypothesis, then repeat the test as backup.

Verify SQLite recovery

This exercise backs up the store’s SQLite data rather than a tar archive. A new /var/backups/shop file must open in SQLite and contain the expected tables. Use readonly mode so a mistyped path cannot create an empty database and hide the problem.

Copy a selected fresh backup to a separate temporary file, then inspect integrity_check and application data. This tests recovery away from the source database. Replace example.sqlite3 with an actual backup filename.

Commands
ls -lh /var/backups/shop
RESTORE=$(mktemp /tmp/shop-restore.XXXXXX.sqlite3)
cp /var/backups/shop/example.sqlite3 "$RESTORE"
sqlite3 -readonly "$RESTORE" "PRAGMA integrity_check; SELECT count(*) FROM orders;"
Example output
ok
2

ok checks structural integrity; the order count checks one application invariant. Also inspect freshness: an old valid backup is not a working new schedule.

Compare with cron’s minimal environment

The backup case’s interactive shell adds the tool directory to PATH. cron uses a different PATH and working directory. Testing as backup with a clean environment reveals dependencies hidden by the administrator’s login session.

In the training console, a successful test creates another backup. Then wait for the next scheduled run: manual success does not prove the schedule or error redirection is correct.

Commands
sudo -u backup env -i HOME=/home/backup SHELL=/bin/bash PATH=/usr/bin:/bin \
  /bin/bash -c 'cd / && /opt/backup/backup.sh'
tail -n 30 /var/log/shop-backup.log

From symptom to check

ObservationNext checkHow to interpret it
Works in the shell, fails in cronPATH, cwd and userReproduce the job’s minimal environment.
Permission deniedParent traversal and destination writesTest access as backup rather than root.
New file exists without expected dataReadonly SQLite and restore testSize and timestamp do not replace structural and application-data checks.

Common mistakes

  • Testing only manually without waiting for cron.
  • Making the directory world-writable to obtain a successful write.
  • Opening a wrong SQLite path without readonly and creating an empty database.

How to prove recovery

  1. Confirm a new backup appears after the repair.
  2. Verify integrity and expected tables in a separate restored database.
  3. Wait for another automatic run as backup.
  4. Preserve source data and appropriate result permissions.

MINI INCIDENT

Manual success is not automation

The root shell creates a backup. A minute later cron reports command not found and creates nothing. What should you compare before changing directory permissions?

Evidence to investigate
interactive PATH: /opt/backup/bin:/usr/bin:/bin
cron PATH: /usr/bin:/bin
backup-tool: command not found
Explore the diagnosis

Compare PATH and how the tool is invoked. Reproduce a clean-environment run as backup. Executable lookup failure does not establish a write-permission problem; after repairing it, still verify permissions and the next scheduled result.

WORKSHOP

Workshop: explain the evidence and choose an action

Manual execution and cron use different conditions. A backup file exists from yesterday.

Sample evidence
manual: backup completed
cron: /opt/backup/backup.sh: line 8: sqlite3: not found
latest backup mtime: yesterday
  1. What is established, and what remains unproven?

    Compare your reasoning

    The manual run finished while automation could not locate the utility. An older file does not establish that the latest scheduled run succeeded. Verify that specific execution.

  2. How can the difference be reproduced?

    Compare your reasoning

    Repeat as the scheduled user with its HOME, minimal PATH and defined directory. Check sqlite3 discovery first, then the actual job and log.

  3. What should supplement the freshness check?

    Compare your reasoning

    Independently restore the selected SQLite backup and check required records. A working schedule, an existing file and usable data are different criteria.

CHECK YOUR UNDERSTANDING

The log contains CMD, but there is no new SQLite backup. What does that establish?

Show explanation

cron attempted to launch the command. Next inspect script errors, directory permissions, paths and the actual output filename. A launch is not a successful backup.

Documentation

Guest progress is stored separately in this browser.

TAKE IT TO THE TERMINAL

Practice on an incident

Restore automatic SQLite backups, verify data in a separate restore and check the next cron run.

DID IT MAKE SENSE?

Was this article helpful?

Rate the article or tell us what needed more explanation. Your feedback helps improve the lessons.

Helpful 0 Not helpful 0

Sign in to rate this article and leave a comment.

Discussion

0

Join the discussion →

No comments yet. Share your first impression or ask a question about this topic.