Skip to content
chronos

Monitor systemd Timers

A systemd timer that fails, or stops firing after a config change, leaves a line in the journal and nothing else. To get an alert, report each run’s result to a Chronos monitor from the service unit.

Add the ping script

Section titled “Add the ping script”

ExecStopPost= runs after the service stops, whether the start succeeded or failed, and systemd passes it the run’s result in $SERVICE_RESULT. The systemd.exec docs recommend it as the place for monitoring hooks.

Save this script as /usr/local/bin/chronos-ping and make it executable with chmod +x:

/usr/local/bin/chronos-ping
#!/bin/sh
# Called from ExecStopPost=, with the monitor's ping URL as $1.
if [ "$SERVICE_RESULT" = "success" ]; then
  exec curl -fsS -m 10 --retry 5 -o /dev/null "$1"
fi
exec curl -fsS -m 10 --retry 5 -o /dev/null \
  --data-raw "result=$SERVICE_RESULT code=$EXIT_CODE status=$EXIT_STATUS" "$1/fail"

On a failure, the body tells you why: result=exit-code code=exited status=2 for a non-zero exit, or result=signal code=killed status=TERM when the process was killed.

Wire up the units

Section titled “Wire up the units”

Create a monitor with the cadence of the timer’s OnCalendar= and the server’s timezone. Then reference the script from the service:

/etc/systemd/system/backup.service
[Unit]
Description=Nightly backup


[Service]
Type=oneshot
ExecStartPre=-/usr/bin/curl -fsS -m 10 --retry 5 -o /dev/null https://ping.chronos.sh/k3Jx9Qm2TnV7bRw4YcZ8dHfL/start
ExecStart=/usr/local/bin/backup.sh
ExecStopPost=-/usr/local/bin/chronos-ping https://ping.chronos.sh/k3Jx9Qm2TnV7bRw4YcZ8dHfL
/etc/systemd/system/backup.timer
[Unit]
Description=Run the nightly backup


[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true


[Install]
WantedBy=timers.target

The ExecStartPre= line sends an optional start ping, so the ping history shows each run’s duration. The - before each ping command’s path keeps a lost ping from failing the service. A lost ping still shows up, because the monitor goes down after the grace period.

Load the units and start the timer:

Terminal window
sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer

Check it works

Section titled “Check it works”

Start the service once by hand with sudo systemctl start backup.service. The monitor moves from new to up. To test the alert, point ExecStart= at /bin/false, run daemon-reload, and start the service again. The down alert shows result=exit-code code=exited status=1.