fix(cron): Normalize timestamps to UTC to prevent double-firing after timezone migration by shanewas · Pull Request #28951 · NousResearch/hermes-agent

shanewas · 2026-05-19T21:44:55Z

PR: Fix Cron Double-Firing After Timezone Migration (#28934)

Summary

Fixes a sticky bug where cron jobs could double-fire after system timezone changes, DST transitions, or config/profile migrations.

Root Cause

_ensure_aware() interpreted naive timestamps using the current system local timezone. After any timezone shift, previously stored last_run_at / next_run_at values were re-interpreted, causing the scheduler to incorrectly consider jobs due again.

Changes

Core Fix (cron/jobs.py): Normalize all cron timestamps to UTC (industry standard).
Migration Helper: Added _migrate_timestamps_to_utc() — safe, idempotent, non-destructive migration for legacy naive timestamps.
Tests: Added TestCronTimezoneMigration covering naive and aware timestamp handling.

Testing

All new tests pass.
Existing cron test suite remains green.
Manual verification: jobs no longer double-fire under simulated timezone shifts.

Security & Standards

No new attack surface (pure datetime normalization).
Follows existing error-handling and logging patterns.
Migration is backward-compatible and non-destructive.
Changes are minimal and highly auditable.

Linked Issue

Closes #28934

Commits on branch

fix(cron): Normalize timestamps to UTC to prevent double-firing after timezone migration
test(cron): Add regression tests for UTC timestamp normalization ([Bug]: Cron jobs double-fire after timezone offset migration #28934)
refactor(cron): Add safe UTC migration helper for legacy timestamps ([Bug]: Cron jobs double-fire after timezone offset migration #28934)

… timezone migration - Changed _ensure_aware() to always use UTC instead of system local time - Legacy naive timestamps are now treated as UTC (safe migration) - This fixes issue NousResearch#28934 where cron jobs would double-fire after DST or timezone changes Refs: NousResearch#28934

…sResearch#28934)

…ousResearch#28934)

teknium1

Thanks for taking a swing at the cron timezone bug. I verified the linked premise against current main, but this implementation does not fix the reported reproducer.

Problems

#28934 is an aware-offset migration bug, not primarily a naive timestamp bug: the issue's repro stores next_run_at=2026-05-19T21:00:00+10:00 and evaluates at 2026-05-19T13:02:00+02:00.
The PR changes cron/jobs.py::_ensure_aware() to normalize aware timestamps to UTC. That still makes the repro due early: 21:00+10 is 11:00Z, and 13:02+02 is 11:02Z, so the comparison remains true.
Current main's due decision is in cron/jobs.py:1124-1125: _get_due_jobs_locked() parses next_run_at, normalizes it, then checks next_run_dt <= now. The fix needs to preserve/rebase cron wall-clock intent at that point.
The added _migrate_timestamps_to_utc() helper in the PR is not called by the diff, so it cannot migrate stored jobs.

Suggested changes

Add a get_due_jobs() regression test using the exact old-offset/new-timezone case from #28934.
Repair recurring cron next_run_at values whose stored offset differs from the current Hermes timezone before dispatching them, and skip that tick.
Drop the unused migration helper unless it is wired into a real runtime path.

This is an automated hermes-sweeper review.

teknium1 · 2026-06-15T14:07:28Z

-    return dt.astimezone(target_tz)
+        # Treat legacy naive timestamps as UTC (migration safety)
+        return dt.replace(tzinfo=timezone.utc)
+    return dt.astimezone(timezone.utc)


Normalizing an old-offset aware timestamp to UTC does not preserve the cron wall-clock intent from #28934. In the reported case, 21:00+10 still becomes due at 13:02+02 after both sides are converted to UTC, so this should be handled in the due-job migration/repair path instead.

teknium1 · 2026-06-15T14:07:28Z

+    return dt.astimezone(timezone.utc)


+def _migrate_timestamps_to_utc(jobs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:


This helper is not called anywhere in the PR diff, so it never migrates persisted last_run_at or next_run_at values at runtime. Either wire it into a real load/repair path with tests, or remove it.

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (#28934, recurrence of #24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from #28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (#28951 normalize-to-UTC, #28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes #28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

@Tranquil-Flow

A recurring cron job persists `next_run_at` as an absolute timestamp with a UTC offset (e.g. `2026-05-19T21:00:00+10:00`). Cron expressions, however, describe *local wall-clock* intent ("run at 21:00"). When Hermes/system timezone changes after the timestamp was persisted, the stored instant is re-interpreted in the new zone: `21:00+10:00` is the instant `13:00+02:00`, which is `<= now` (13:02+02:00) — so the job fires HOURS EARLY, then `compute_next_run` advances it via croniter to `21:00+02:00` the same day, producing a SECOND fire. (NousResearch#28934, recurrence of NousResearch#24289.) `_get_due_jobs_locked` now detects this precise migration case before the due check: for a `cron` job whose converted instant looks due, whose stored UTC offset differs from the current zone's, AND whose stored *wall-clock* time is still in the future (distinguishing a migrated offset from a genuinely missed run), it recomputes `next_run_at` from the schedule and skips the early fire — preserving the local wall-clock intent. Verified against the issue's reproducer: stored `21:00+10` under runtime `+02:00` at wall-clock `13:02` is rescheduled to `21:00+02` instead of firing early + again. Salvaged from NousResearch#28941 by @Tranquil-Flow (authorship preserved). Chosen over the alternative approaches (NousResearch#28951 normalize-to-UTC, NousResearch#28985 rebase-and-match) because UTC-normalization does not change the absolute-instant comparison and so does not fix the early fire, and this guard is the tightest: it only acts when all four conditions hold and reuses the existing `compute_next_run`. Fixes NousResearch#28934

shanewas added 3 commits May 20, 2026 06:30

test(cron): Add regression tests for UTC timestamp normalization (Nou…

74c8a87

…sResearch#28934)

refactor(cron): Add safe UTC migration helper for legacy timestamps (N…

956185c

…ousResearch#28934)

alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cron Cron scheduler and job management labels May 19, 2026

alt-glitch mentioned this pull request May 20, 2026

fix(cron): repair migrated timezone offsets #28985

Closed

3 tasks

Jiahui-Gu mentioned this pull request May 20, 2026

fix(cron): prevent double-fire after timezone offset migration (#28934) #29218

Closed

2 tasks

teknium1 reviewed Jun 15, 2026

View reviewed changes

kshitijk4poor mentioned this pull request Jun 21, 2026

fix(cron): repair migrated timezone offsets to prevent double-fire #50034

Merged

kshitijk4poor closed this in #50034 Jun 21, 2026

JoaoMarcos44 mentioned this pull request Aug 17, 2026

fix(cron): record the timezone a cron schedule is evaluated in #88581

Open

4 tasks

Navigation Menu

Uh oh!

fix(cron): Normalize timestamps to UTC to prevent double-firing after timezone migration - #28951

fix(cron): Normalize timestamps to UTC to prevent double-firing after timezone migration#28951
shanewas wants to merge 3 commits into
NousResearch:mainfrom
shanewas:fix/cron-timezone-double-fire

shanewas commented May 19, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jun 15, 2026

Uh oh!

teknium1 Jun 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		return dt.astimezone(timezone.utc)


		def _migrate_timestamps_to_utc(jobs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:

Uh oh!

Conversation

shanewas commented May 19, 2026

PR: Fix Cron Double-Firing After Timezone Migration (#28934)

Summary

Root Cause

Changes

Testing

Security & Standards

Linked Issue

Commits on branch

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants