Laravel schedules that randomly stop: withoutOverlapping() and its 24-hour lock
A short note so I don't make this mistake again.
The symptom: a scheduled command that runs every few minutes just stops. No errors, no failed jobs, and
schedule:run still prints it as Running ... DONE. A day later it starts working again on its own.
The cause
The culprit is ->withoutOverlapping() on the schedule definition.
withoutOverlapping() takes a cache lock before the command runs and releases it when the command
finishes. Its signature (Laravel 13):
public function withoutOverlapping($expiresAt = 1440, $releaseOnTerminationSignals = true)
$expiresAt is in minutes, so the default lock lives for 1440 minutes, or 24 hours.
The lock is released in finish(), or on SIGTERM/SIGINT/SIGQUIT if pcntl is loaded. It is not
released when the process dies in a way PHP can't catch:
SIGKILL: the OOM killer, or Kubernetes killing the pod after the termination grace period during a deploy or a node drain- a container or server that just goes away
- a
runInBackground()task killed before its trailingschedule:finishcall runs (background tasks don't get the signal handlers at all)
When that happens, the lock stays in the cache (Redis in my case) until the TTL runs out. Every
schedule:run in the meantime sees the lock, assumes the previous run is still going, and skips the
task. A task that runs every five minutes misses 288 runs.
It's easy to miss because the skip is almost invisible. schedule:run still dispatches
ScheduledTaskStarting and ScheduledTaskFinished and prints the task as done. The overlap check
happens inside Event::run(), which just returns early.
If you suspected onOneServer() too, it's not the problem. Its lock key includes the minute the task
was due (mutexName().Hi) and lives for an hour, so a stale one can't block the next run. It doesn't
take a TTL argument at all.
I don't know about you, but I'd been adding ->withoutOverlapping() to longer-running tasks on
autopilot, without thinking much about how it works or what can go wrong.
The fix
Always pass a TTL a bit longer than the longest run you expect, and shorter than the gap you're willing to lose:
// routes/console.php
Schedule::command('reports:sync')
->everyFiveMinutes()
->withoutOverlapping(10) // minutes
->onOneServer();
Schedule::command('backups:run')
->dailyAt('03:00')
->withoutOverlapping(120)
->onOneServer();
If the process is killed, the task now comes back within minutes instead of a day.
That said, a daily task like backups:run probably doesn't need ->withoutOverlapping() at all. It
would have to run for a full day to overlap with itself, and onOneServer() already stops two servers
from running it at the same time.
A few more things that help:
- Give the pod a
terminationGracePeriodSecondslong enough for scheduled tasks to finish or exit cleanly afterSIGTERM, before Kubernetes falls back toSIGKILL. - Make sure
pcntlis installed in the scheduler image. Without it, evenSIGTERMleaves the lock behind. - Clear stuck locks by hand with
php artisan schedule:clear-cache. It's worth running as part of a deploy, after the old scheduler is gone. - Monitor that the task actually did something, such as a heartbeat ping at the end of the command or a "last run" timestamp, rather than relying on the scheduler's output.
A test to keep it that way
It's an easy argument to forget, so let a test catch it. This Pest test boots the app, walks every
registered scheduled task, and fails if any of them uses withoutOverlapping() with the default
24-hour lock:
<?php
use Illuminate\Console\Scheduling\Schedule;
it('gives every withoutOverlapping() task an explicit lock TTL', function () {
$tasks = collect(app(Schedule::class)->events())
->filter(fn ($event) => $event->withoutOverlapping && $event->expiresAt === 1440)
->map(fn ($event) => $event->getSummaryForDisplay())
->values();
expect($tasks)->toBeEmpty(
'These tasks use withoutOverlapping() with the default 24-hour lock: '.$tasks->implode(', ')
);
});
It inspects the schedule at runtime instead of grepping the source, so it also covers tasks registered
in bootstrap/app.php or a service provider. It can't tell an empty withoutOverlapping() apart from an
explicit withoutOverlapping(1440), but a 24-hour lock is what we want to avoid either way.