feat: route new task runs to a parallel task_run_v2 table#4000
Draft
d-cs wants to merge 90 commits into
Draft
Conversation
Replaces the seven throwing stubs on PostgresRunStore with verbatim relocations of the Prisma statements from runAttemptSystem: startAttempt, completeAttemptSuccess, recordRetryOutcome, requeueRun, recordBulkActionMembership, cancelRun, and failRunPermanently. Each method splices the caller-supplied select/include into the Prisma call. Tests use real Postgres containers and cover each method including edge cases (append semantics, conditional fields in cancelRun).
…y-clear, and array-append methods
Replaces the seven throwing stubs in PostgresRunStore with verbatim-relocated
Prisma statements sourced from delayedRunSystem, debounceSystem, updateMetadata,
idempotencyKeys, resetIdempotencyKey, batchTriggerV3, and the realtime-stream
route handlers.
- rescheduleRun: writes delayUntil always; queueTimestamp when provided; nested
DELAYED executionSnapshot when snapshot arg provided
- enqueueDelayedRun: sets status PENDING + queuedAt
- rewriteDebouncedRun: pass-through update with associatedWaitpoint include
- updateMetadata: optimistic-lock path (updateMany with version predicate) or
direct path (update without predicate); both return { count }
- clearIdempotencyKey: three discriminated-union branches — byId clears both
columns, byPredicate clears both, byFriendlyIds clears only idempotencyKey
- pushTags: push-append to runTags array; returns { updatedAt }
- pushRealtimeStream: push-append to realtimeStreams array; returns void
…bapp BaseService Add RunStore field to SystemResources, instantiate PostgresRunStore in RunEngine constructor (after prisma/readOnlyPrisma are set), and expose it on the resources object passed to all systems. Create a webapp singleton (runStore.server.ts) and thread it as a default parameter into BaseService so subclasses can access it without changes.
…eave-unchanged semantics)
…s through RunStore
… debounce writes through RunStore
… writes through RunStore
The service statically imported the db.server-backed runStore singleton, which dragged the Prisma client into otherwise-light test module graphs and opened an eager connection to DATABASE_URL on import. The metadata service test then threw an unhandled connection error whenever no database was reachable at the configured address. Make runStore a required constructor option, pass the singleton at the production construction site, and inject a testcontainer-backed store in the tests.
Add findRun, findRunOrThrow and findRuns to RunStore, mirroring the existing write methods. They pass where/select/include through the same Prisma generics and default to the read replica, while letting the caller pass the writer or a transaction client when needed. This lets Postgres reads of TaskRun be routed through the store the same way writes already are. Additive only; no call sites change yet.
Add a no-args overload to findRun, findRunOrThrow and findRuns that returns the whole TaskRun row, for callers that read a run without a select or include.
Relocate the direct TaskRun reads in the engine and its systems to the RunStore read methods, preserving the exact client (writer, replica, or transaction) at each site. Behavior-preserving; the engine test suite is unchanged.
…tore Relocate the direct TaskRun reads in webapp services, run-engine concerns, realtime, mollifier and metadata to the RunStore read methods, preserving the exact client (writer, replica, or transaction) at each site. The run hydrator now receives the store by injection. Behavior-preserving.
Relocate the dashboard presenter TaskRun reads to the RunStore read methods, preserving the exact client per site. Behavior-preserving.
…store Relocate the route and loader TaskRun reads to the RunStore read methods, preserving the exact client per site, including the replica-resolve then writer-recheck realtime paths. Behavior-preserving.
…store Decompose the three reads that pulled TaskRun in through a parent model's relation include (alert, batch results, attempt dependencies): query the parent without the include, hydrate the run(s) via RunStore in a single batched read, and stitch them back. Preserves field selection, ordering, null handling and the query client. Adds container-backed tests for the batch-results and cancel-dependencies paths.
…tover The recovery script joins TaskRunExecutionSnapshot to TaskRun in raw SQL, so it is the one TaskRun read not routed through the run store. Add a note to revisit it at table cutover.
![devin-ai-integration[bot]](https://avatars-githubusercontent-com.shimmerl.top/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The two-table shape merge could leave one upstream fetch pending without a rejection handler when it aborts the race loser or rethrows from the catch block. Attach a detached no-op catch to both fetches up front so an abandoned fetch can never surface as an unhandled rejection on any path. Also document that a tag/batch subscription opens two upstream Electric connections while an org spans both run tables.
…ectric shapes Electric realtime shapes are bound to a single table, so a task_run_v2 run was invisible to realtime subscriptions. The previous approach merged two Electric shapes per tag/batch feed under a composite cursor, which doubled Electric long-poll connections for those feeds. Electric is being retired in favor of the native realtime backend, which is table-agnostic and already observes both run tables, so that merge is throwaway. Drop the Electric dual-shape merge (revert realtimeClient to its single-table form, remove the merge module) and instead gate runTableV2 on the native backend: a run only routes to task_run_v2 when the deployment has native realtime enabled and the org's realtimeBackend flag is native. This keeps v2 runs realtime-observable without touching Electric, and the gate auto-satisfies once Electric is removed and native is the default. The idempotency pre-gate claim inherits the same gate.
Completes the Electric-merge removal: a run only routes to task_run_v2 when the deployment has native realtime enabled and the org's realtimeBackend flag is native. Electric shapes are single-table and can't observe a v2 run, so without this gate a v2 run would be realtime-invisible. shouldUseV2RunTable takes the native-realtime master switch as a parameter (kept env-free for unit tests); the trigger mint site and the idempotency pre-gate claim both pass it.
Restore the both-table Electric shape merge so tag-list and batch realtime feeds observe runs in TaskRun and task_run_v2 together, and gate the v2 run table on the runTableV2 flag alone (drop the native-realtime coupling). New runs route to task_run_v2 whenever an org has the flag on and stay visible in realtime on the existing Electric backend. Single-run feeds route to one table by id format; only tag and batch feeds fan out to both shapes under one composite continuation.
…ed window Routes that walk the run hierarchy through a Prisma relation only see one physical table, so during a runTableV2 flag flip (a parent and child on opposite tables) they silently miss the cross-table run. This closes the reachable cases: - cancelRun resolves child runs across both tables, so cancelling a parent cascades to a child in the other table instead of leaving it executing and holding concurrency. - updateMetadata routes metadata.parent/root operations to the scalar parent/root id, so they reach a parent in the other table instead of falling back to the child run. - a one-time-use token with no idempotency key now takes a cross-table claim for v2 orgs, so two presentations straddling a flip cannot each mint a run in a different table. - the Electric shape merge reports up-to-date only when both tables are caught up, so a multi-chunk initial snapshot no longer drops the rows that arrive after the first chunk.
… mixed window A cuid parent (TaskRun) with a ksuid child (task_run_v2): cancelling the parent must cascade to the child in the other table. Fails against the old table-bound childRuns relation, passes with the cross-table findRuns lookup.
…tables An unordered take capped each run table independently and concatenated the two results, so a both-table read could silently drop one table rows once the other filled the cap. Reject it like the existing skip and cursor guards; callers that need a bounded cross-table read pass an orderBy for the keyset merge.
The guard added in the previous commit makes that call throw rather than return a non-deterministic cap; this test asserted the removed cap behavior. The throw is covered by the guard test alongside the skip/cursor guards.
…-token claim) - The strengthened findRuns guard threw on GET /api/v1/runs/:runId/spans/:spanId, which pages child runs with take and no orderBy across both tables. Add a createdAt order so it takes the bounded cross-table merge (and the 50-row cap is now deterministic, most recent first) instead of throwing for every org. - Key the one-time-use-token cross-table claim on the token alone (a reserved task slot), matching the task-independent oneTimeUseToken unique constraint, so a multi-task token cannot mint twice across the flip. Stop excluding triggerAndWait from the token claim. Always resolve a held claim on the success path (publish, else release) so it cannot leak until its TTL.
… shape merge The Electric dual-shape merge was a bridge to let the Electric backend observe v2 runs during the cutover, but Electric is short-lived and the merge taxed every tag/batch realtime feed with a second long-poll the moment it deployed. Gate the v2 run table on the native realtime backend instead (the native client is table-agnostic and observes v2 runs directly), so a run only routes to task_run_v2 once its org is on native. Remove the merge module and restore the single-table Electric proxy. The cross-table correctness work stays: a v2 run can still have a cross-table parent or child once an org flips, so the cancelRun cascade, metadata parent/root routing, the one-time-token claim, and the findRuns guard all still apply regardless of realtime backend.
…v2 orgs, add cross-table tests The idempotency-key dedup is a non-id predicate, so RunStore read BOTH run tables in parallel on every idempotency-keyed trigger, including orgs not cut over to v2 (whose runs only live in TaskRun, so the task_run_v2 query is always empty; while native realtime is off that is every org). Add an optional `tables: "legacy" | "both"` scope to findRun and pass "legacy" from the idempotency concern when the org is not on v2, keeping the trigger hot path single-table. Backfills cross-table tests the audit flagged as missing: findRun legacy-scope skips task_run_v2, and clearIdempotencyKey fans out across both tables (byPredicate hits v2; a mixed byFriendlyIds array clears both).
…close remaining cross-table sites Make the v2 path both correct and performant for turning the flag on: - findRuns gains an optional tables: "legacy" | "both" scope (mirroring findRun), threaded through hydrateChildRuns/hydrateParentAndRoot. While an org is not on v2 its runs only live in TaskRun, so callers pass "legacy" to skip the empty task_run_v2 query. - ApiRetrieveRun resolves parent/root and children in parallel (one round-trip instead of two) and scopes the reads to legacy for non-v2 orgs, so the public run-retrieve no longer pays an extra both-table query on every call. - The run-inspector side panel resolves parent/root by id across both tables (was a table-bound relation select that returned null for a cross-table parent in the mixed window). - recover-stuck-runs joins TaskRunExecutionSnapshot against TaskRun UNION task_run_v2 so a stuck v2 run is found and re-enqueued.
…un tables The two realtime v1-streams routes (read/create + append) resolved a target:"parent"|"root" via a table-bound parentTaskRun/rootTaskRun relation select, which returns null for a cross-table parent/root in the runTableV2 mixed window and 404s a target that exists. Select the scalar parentTaskRunId/rootTaskRunId and resolve the target by id through RunStore (routes by id format), matching the presenter fixes. The "self" target is unchanged.
…e per-org flag A run's physical table is fixed by its id format, not an org's current runTableV2 flag. An org that was on v2 then flipped the flag off still holds v2 runs (they stay readable, routed by id), so scoping the dedup and hierarchy reads to "legacy" off the per-org flag would miss those v2 runs: it would silently drop a v2 run's children and parent on retrieve, and let a duplicate through idempotency dedup. Gate the scope on whether ANY v2 run can exist in the deployment (the native realtime master switch) instead. While native is off no v2 run exists anywhere, so "legacy" is safe and still skips the empty task_run_v2 query; once native is on, every read covers both tables.
…py window Forward-looking robustness: - recover-stuck-runs joins TaskRun UNION (not UNION ALL) task_run_v2, so if a later copy step leaves a run briefly in both tables under the same id the identical clones collapse to one row and DISTINCT ON stays unambiguous. - Document that the findRuns "legacy" scope hint is for non-id predicates; an id read already routes by format, so the hint is redundant there (and a KSUID-only id predicate scoped legacy correctly returns nothing).
State that no v2 run exists "until native is enabled" (minting requires it) rather than the absolute "while native is off", which is not true after a deployment-wide native rollback. Comment-only.
…2 enable guard, FK-drift guard Make task_run_v2 a faithful (non-degraded) clone of TaskRun and make the v2 cutover safe to actually enable: - Add the INCLUDE (id) WITH (fillfactor=90) covering clause to task_run_v2's (runtimeEnvironmentId, createdAt DESC) index so the dashboard run-list query keeps index-only scans, matching TaskRun. Columns were already identical; v2 now has every index TaskRun has (plus a (createdAt, id) keyset index it needs for the cross-table cursor merge). - Reject enabling runTableV2 for an org unless realtimeBackend is "native" (validateFeatureFlagInvariants), wired into both admin feature-flag write routes. A v2 run minted while the org is still on Electric is realtime invisible, so block the bad combination at write time, not just at read time. - Add a guard test that fails CI if a generated migration re-adds an incoming foreign key to TaskRun (after the decoupling drop) or adds one referencing task_run_v2: the Prisma drift footgun that would re-couple the tables or break cross-table run creation.
The child models (TaskRunAttempt, TaskRunWaitpoint, TaskRunDependency, BatchTaskRunItem, Checkpoint, CheckpointRestoreEvent, TaskRunExecutionSnapshot, BulkActionItem) declared their taskRunV2/runV2 mirror relation as non-nullable, but a child row references a run in exactly one table, so the mirror resolves to null for any row whose run is in the other table. An include of the mirror would return null under a non-null type. Type them as TaskRunV2? to match runtime: the legacy relation anchors the required scalar, so Prisma accepts the optional second relation. Client-type change only (the FK is already stripped and the scalar unchanged), so no migration.
Prisma emits foreign keys as REFERENCES "public"."TaskRun" (schema-qualified) in every generated migration in this repo, but the guard only matched the bare "TaskRun"https://github.com/"task_run_v2", so it was effectively a no-op against real migrate-dev output: it would not catch a regenerated _v2_fkey, nor the implicit m2m _WaitpointRunConnections_A_fkey / _TaskRunToTaskRunTag_A_fkey (the former would break trigger-and-wait for v2 runs). Match both bare and schema-qualified forms, with fixtures pinning both so the qualified form cannot regress undetected.
triggerFailedTask minted the pre-failed run with a cuid (RunId.generate), so it landed in legacy TaskRun even for an org cut over to v2. Trigger-time failures (queue limits, validation, payload errors) are common for some orgs, and these runs frequently carry a parentTaskRunId / resumeParentOnCompletion / batch, so each one created an ongoing cross-table edge (a v2 parent or batch with a legacy failed child) on the failure path, not just the transient mixed window. Gate the id mint on shouldUseV2RunTable like triggerTask: the main call() path uses the request org flags; the degraded callWithoutTraceEvents() path loads them by org id and falls back to a legacy id only if the org cannot be resolved.
…ope reads on v2-may-exist #1 publication-readiness interlock: a v2 run minted before task_run_v2 is in the ClickHouse replication publication is permanently absent from ClickHouse (Postgres only decodes changes for transactions that begin after ALTER PUBLICATION ADD TABLE, which the replication leader runs at its own startup, not via a migration), and the run list/metrics/tags are ClickHouse-only. Add a cached, periodically-refreshed status (runTableV2Status.server.ts) and gate minting (triggerTask, triggerFailedTask) through canMintV2Run = org cut over AND table published. Minting fails safe to legacy until the publication carries the table and self-heals once it does, removing the manual pg_publication_tables enable step. #2 native-rollback read scope: cross-table read scoping (idempotency dedup, ApiRetrieveRun hierarchy) keyed on the native master switch alone, so disabling native realtime after v2 runs exist re-scoped reads to legacy and hid existing v2 runs (an idempotency dedup miss means duplicate execution). Scope on v2RunsMayExist (native on OR task_run_v2 has rows) instead; it is monotonic, so the read scope cannot regress once v2 runs exist.
…lify publication probe by schema Code-review follow-ups on the v2 hardening: - runTableV2Status no longer starts its background poller under vitest (NODE_ENV=test). The module is imported by the mint/read sites, so the import-time poll plus setInterval was firing live DB queries against the test database and leaking a timer, and the async refresh could race tests that drive the cached status directly. Tests exercise the gates by mutating the cached state, so the poller only gets in the way. - The publication-readiness probe now filters pg_publication_tables on schemaname = "public", so a same-named table in another published schema cannot satisfy the check.
![coderabbitai[bot]](https://avatars-githubusercontent-com.shimmerl.top/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 84ea2db4-632f-40e2-8122-52e73c869e77
📒 Files selected for processing (35)
apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.tsapps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/routes/resources.runs.$runParam.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/app/v3/mollifier/idempotencyClaim.server.tsapps/webapp/app/v3/mollifier/mollifierBuffer.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/test/mollifierClaimResolution.test.tsapps/webapp/test/mollifierResetIdempotencyKey.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsapps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/updateMetadata.test.tsinternal-packages/database/prisma/migrations/20260623090000_task_run_v2_covering_index/migration.sqlinternal-packages/database/prisma/schema.prismainternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsinternal-packages/run-store/src/PostgresRunStore.test.tsinternal-packages/run-store/src/PostgresRunStore.tsinternal-packages/run-store/src/types.tsscripts/recover-stuck-runs.ts
✅ Files skipped from review due to trivial changes (1)
- internal-packages/database/prisma/migrations/20260623090000_task_run_v2_covering_index/migration.sql
🚧 Files skipped from review as they are similar to previous changes (11)
- apps/webapp/test/mollifierClaimResolution.test.ts
- apps/webapp/test/mollifierResetIdempotencyKey.test.ts
- apps/webapp/app/presenters/v3/ApiRetrieveRunPresenter.server.ts
- apps/webapp/test/runTableV2.test.ts
- apps/webapp/app/v3/mollifier/idempotencyClaim.server.ts
- apps/webapp/app/presenters/v3/RunPresenter.server.ts
- apps/webapp/app/v3/mollifier/mollifierBuffer.server.ts
- internal-packages/run-store/src/PostgresRunStore.test.ts
- apps/webapp/app/presenters/v3/SpanPresenter.server.ts
- internal-packages/database/prisma/schema.prisma
- internal-packages/run-store/src/PostgresRunStore.ts
📜 Review details
🧰 Additional context used
📓 Path-based instructions (13)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects insteadImport from
@trigger.dev/sdkwhen writing Trigger.dev tasks. Never use@trigger.dev/sdk/v3or deprecatedclient.defineJob
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
**/*.{ts,tsx,js,jsx}: Prefer static imports over dynamic imports. Only use dynamicimport()when circular dependencies cannot be resolved, code splitting is needed for performance, or the module must be loaded conditionally at runtime
Import subpaths only frompackages/core(@trigger.dev/core), never import from the root
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use vitest for all tests in the Trigger.dev repository
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: Access environment variables through theenvexport ofenv.server.tsinstead of directly accessingprocess.env
Use subpath exports from@trigger.dev/corepackage instead of importing from the root@trigger.dev/corepathUse named constants for sentinel/placeholder values (e.g.
const UNSET_VALUE = '__unset__') instead of raw string literals scattered across comparisons
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
apps/webapp/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Do not import
env.server.tsdirectly or indirectly into test files; instead pass environment-dependent values through options/parameters to make code testableFor testable code, never import
env.server.tsin test files. Pass configuration as options instead (e.g.,realtimeClient.server.tstakes config as constructor arg,realtimeClientGlobal.server.tscreates singleton with env config)
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.test.{ts,tsx}: Never mock anything in tests - use testcontainers instead
Test files should be placed next to source files (e.g.,MyService.ts->MyService.test.ts)
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
**/*.{js,ts,tsx,jsx,css,json,md}
📄 CodeRabbit inference engine (AGENTS.md)
Use Prettier for code formatting and run
pnpm run formatbefore committing
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
**/*.test.{js,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.test.{js,ts,tsx}: Test files should live beside the files under test and use descriptivedescribeanditblocks
Use vitest for unit testing
Tests should avoid mocks or stubs and use helpers from@internal/testcontainerswhen Redis or Postgres are needed
Files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
internal-packages/run-engine/src/engine/tests/**/*.test.ts
📄 CodeRabbit inference engine (internal-packages/run-engine/CLAUDE.md)
Implement tests for RunEngine in
src/engine/tests/using testcontainers for Redis and PostgreSQL containerization
Files:
internal-packages/run-engine/src/engine/tests/cancelling.test.ts
apps/webapp/**/*.server.ts
📄 CodeRabbit inference engine (apps/webapp/CLAUDE.md)
apps/webapp/**/*.server.ts: Never userequest.signalfor detecting client disconnects. UsegetRequestAbortSignal()fromapp/services/httpAsyncStorage.server.tsinstead, which is wired directly to Expressres.on('close')and fires reliably
Access environment variables viaenvexport fromapp/env.server.ts. Never useprocess.envdirectly
Always usefindFirstinstead offindUniquein Prisma queries.findUniquehas an implicit DataLoader that batches concurrent calls and has active bugs even in Prisma 6.x (uppercase UUIDs returning null, composite key SQL correctness issues, 5-10x worse performance).findFirstis never batched and avoids this entire class of issues
Files:
apps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
internal-packages/run-engine/src/engine/systems/**/*.ts
📄 CodeRabbit inference engine (internal-packages/run-engine/CLAUDE.md)
Integrate OpenTelemetry tracer and meter instrumentation in RunEngine systems for observability
Files:
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts
🧠 Learnings (19)
📚 Learning: 2026-03-22T13:26:12.060Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3244
File: apps/webapp/app/components/code/TextEditor.tsx:81-86
Timestamp: 2026-03-22T13:26:12.060Z
Learning: In the triggerdotdev/trigger.dev codebase, do not flag `navigator.clipboard.writeText(...)` calls for `missing-await`/`unhandled-promise` issues. These clipboard writes are intentionally invoked without `await` and without `catch` handlers across the project; keep that behavior consistent when reviewing TypeScript/TSX files (e.g., usages like in `apps/webapp/app/components/code/TextEditor.tsx`).
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-03-22T19:24:14.403Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3187
File: apps/webapp/app/v3/services/alerts/deliverErrorGroupAlert.server.ts:200-204
Timestamp: 2026-03-22T19:24:14.403Z
Learning: In the triggerdotdev/trigger.dev codebase, webhook URLs are not expected to contain embedded credentials/secrets (e.g., fields like `ProjectAlertWebhookProperties` should only hold credential-free webhook endpoints). During code review, if you see logging or inclusion of raw webhook URLs in error messages, do not automatically treat it as a credential-leak/secrets-in-logs issue by default—first verify the URL does not contain embedded credentials (for example, no username/password in the URL, no obvious secret/token query params or fragments). If the URL is credential-free per this project’s conventions, allow the logging.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma error P1001 ("Can't reach database server") in TypeScript, don’t assume a single error shape. Prisma can surface P1001 via two different error classes/fields: `PrismaClientKnownRequestError` exposes it as `err.code === "P1001"` (common during mid-query connection drops), while `PrismaClientInitializationError` exposes it as `err.errorCode === "P1001"` (common on client startup failure). Therefore, predicates should use `err.code === "P1001" || err.errorCode === "P1001"`. Do not flag `err.code === "P1001"` as “unreachable/never matches,” as it is expected in production.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma errors for P1001 ("Can't reach database server"), do not assume it only appears under a single property name. Prisma may surface P1001 via either `PrismaClientKnownRequestError` (`err.code === "P1001"`, e.g., mid-query connection drops) or `PrismaClientInitializationError` (`err.errorCode === "P1001"`, e.g., client startup connection failure). To reliably detect the condition, check `err.code === "P1001" || err.errorCode === "P1001"`, and avoid review rules that would incorrectly flag `err.code === "P1001"` as unreachable/never-matching.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-06-13T19:53:13.759Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3937
File: packages/trigger-sdk/skills/realtime-and-frontend/SKILL.md:258-260
Timestamp: 2026-06-13T19:53:13.759Z
Learning: When reviewing code that uses `trigger.dev/react-hooks`’s `useRealtimeRun`, preserve the call signature where the first argument is the full realtime handle object (not `handle.id`). This is intentional to maintain type-safety and is consistent with the official docs; do not suggest changing the first argument from the handle object to `handle.id`.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-06-17T17:13:49.929Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3948
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.bulk-actions.$bulkActionParam/route.tsx:48-62
Timestamp: 2026-06-17T17:13:49.929Z
Learning: In triggerdotdev/trigger.dev, within `dashboardLoader`/`dashboardAction` (or similar context resolver code) whenever you resolve an organization ID from an organization slug for RBAC/enterprise authorization scope, always read from the primary Prisma client (`prisma`), not `$replica`. Using `$replica` can hit replica-lag and cause the RBAC lookup/authorization to run without the correct org scope (bypassing intended role enforcement). Implement the slug→org lookup with `prisma.organization.findFirst(...)` (or equivalent primary-client query) and add an inline comment documenting why the primary client is required (replica lag could lead to unscoped RBAC checks).
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-05-07T12:25:18.271Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3531
File: apps/webapp/test/sentryTraceContext.server.test.ts:9-47
Timestamp: 2026-05-07T12:25:18.271Z
Learning: In the triggerdotdev/trigger.dev webapp test suite, it is acceptable to leave `createInMemoryTracing()` calls that register a global `NodeTracerProvider` without `afterEach`/`afterAll` teardown. Do not flag this as a test-ordering risk when the code follows the established pattern used across webapp tests (e.g., replication service/benchmark/backfiller tests). This is considered safe because `trace.getActiveSpan()` when called outside a `context.with(...)` block reads `AsyncLocalStorage.getStore()` (undefined when no `run()` scope exists), so it falls back to `ROOT_CONTEXT` with no attached span—regardless of which provider is registered.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
📚 Learning: 2026-05-28T20:02:10.647Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 3772
File: apps/webapp/test/findOrCreateBackgroundWorker.test.ts:1-1
Timestamp: 2026-05-28T20:02:10.647Z
Learning: In the triggerdotdev/trigger.dev monorepo, for the `apps/webapp` package use the established convention of storing Vitest tests (unit, integration, and e2e) under `apps/webapp/test/` rather than colocating them next to source files. Do not flag files located in `apps/webapp/test/` as violating any rule that says to colocate tests with source.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
📚 Learning: 2026-05-12T21:04:05.815Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3542
File: apps/webapp/app/components/sessions/v1/SessionStatus.tsx:1-3
Timestamp: 2026-05-12T21:04:05.815Z
Learning: In this Remix + TypeScript codebase, do not flag a server/client boundary violation when a file imports only types from a module matching `*.server`.
Specifically, it’s safe to import types using `import type { Foo } from "*.server"` or `import { type Foo } from "*.server"` because TypeScript erases type-only imports at compile time and they emit no JavaScript, so they won’t cross the Remix server/client bundle boundary.
Only raise the boundary concern for value imports (e.g., `import { Foo }` without `type`, or `import Foo`), since those produce JavaScript output.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-05-18T14:40:02.173Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3658
File: packages/core/src/v3/realtimeStreams/manager.test.ts:1-147
Timestamp: 2026-05-18T14:40:02.173Z
Learning: In the triggerdotdev/trigger.dev repo, the policy “Never mock anything — use testcontainers instead” should only be enforced for integration tests that interact with real external services (e.g., Redis, Postgres) via actual infrastructure. For unit tests that exercise pure in-memory logic (e.g., cache semantics) it is OK to stub collaborators such as `ApiClient` using Vitest (`vi.fn()`) to assert call counts or control behavior. Do not flag `vi.fn()`-based `ApiClient` stubs in unit tests as violations of the testcontainers policy.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
📚 Learning: 2026-06-04T18:16:35.386Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 3836
File: apps/supervisor/src/backpressure/backpressureMonitor.ts:3-5
Timestamp: 2026-06-04T18:16:35.386Z
Learning: When reviewing TypeScript in this repo, apply the rule “prefer type aliases over interfaces” only to data/object shapes and union/intersection type modeling. If an interface is being used as a behavioral contract for collaborators to implement (e.g., method-shape interfaces that define required behavior, such as `BackpressureLogger` / `BackpressureSignalSource` in `apps/supervisor/src/backpressure/backpressureMonitor.ts`), keep it as an `interface` and do not flag it as a type-alias-vs-interface violation.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-06-09T17:58:04.699Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 3879
File: apps/webapp/app/models/vercelIntegration.server.ts:619-630
Timestamp: 2026-06-09T17:58:04.699Z
Learning: In this codebase, outbound raw `fetch` calls should typically rely on Node/undici’s default request timeout (about ~300s) rather than adding a per-call `AbortController` + `setTimeout` wrapper inside individual functions (e.g. in files like `apps/webapp/app/models/vercelIntegration.server.ts`). During code review, do not flag the absence of a per-call timeout on a single `fetch` as an issue; if per-call timeouts are needed, they should be implemented via a codebase-wide convention (e.g., a shared fetch wrapper or documented pattern) rather than ad-hoc per-function changes.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsapps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.tsapps/webapp/app/routes/resources.runs.$runParam.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.tsapps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsinternal-packages/run-store/src/types.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/v3/featureFlags.tsapps/webapp/test/oneTimeUseTokenClaim.test.tsscripts/recover-stuck-runs.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-06-16T09:19:47.637Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3960
File: apps/webapp/test/prismaInfrastructureErrorCapture.test.ts:0-0
Timestamp: 2026-06-16T09:19:47.637Z
Learning: In this repo’s Vitest setup, `vitest.config.ts` uses `globals: true`, so identifiers like `vi`, `describe`, `it`, and `expect` are available as globals in Vitest test files. During code review, do not flag missing `vi`/`describe`/`it`/`expect` imports as a runtime error or correctness issue when they’re used in `*.test.ts/tsx` or `*.spec.ts/tsx` files. Explicit imports are still preferred for consistency, but they’re not required for runtime behavior.
Applied to files:
apps/webapp/test/runTableFkDriftGuard.test.tsapps/webapp/test/runTableV2Status.test.tsapps/webapp/test/featureFlagInvariants.test.tsinternal-packages/run-engine/src/engine/tests/cancelling.test.tsapps/webapp/test/updateMetadata.test.tsapps/webapp/test/oneTimeUseTokenClaim.test.ts
📚 Learning: 2026-03-29T19:16:28.864Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 3291
File: apps/webapp/app/v3/featureFlags.ts:53-65
Timestamp: 2026-03-29T19:16:28.864Z
Learning: When reviewing TypeScript code that uses Zod v3, treat `z.coerce.*()` schemas as their direct Zod type (e.g., `z.coerce.boolean()` returns a `ZodBoolean` with `_def.typeName === "ZodBoolean"`) rather than a `ZodEffects`. Only `.preprocess()`, `.refine()`/`.superRefine()`, and `.transform()` are expected to wrap schemas in `ZodEffects`. Therefore, in reviewers’ logic like `getFlagControlType`, do not flag/unblock failures that require unwrapping `ZodEffects` when the input schema is a `z.coerce.*` schema.
Applied to files:
apps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/v3/featureFlags.ts
📚 Learning: 2026-06-09T16:27:26.195Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 3878
File: apps/webapp/app/v3/services/computeTemplateCreation.server.ts:0-0
Timestamp: 2026-06-09T16:27:26.195Z
Learning: When working in triggerdotdev/trigger.dev code related to worker-group/region default resolution (e.g., defaultWorkerInstanceGroupId handling used by getGlobalDefaultWorkerGroup, getDefaultWorkerGroupForProject, and RegionsPresenter), do NOT add org-level featureFlags overrides in only one resolution site. That can cause template creation routing/decisions to diverge from actual run routing. If org-level override of the default region/worker group is required, it must be centralized in getGlobalDefaultWorkerGroup so every resolution path remains aligned.
Applied to files:
apps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/v3/featureFlags.ts
📚 Learning: 2026-05-05T09:38:02.512Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3523
File: apps/webapp/app/routes/api.v3.batches.ts:178-181
Timestamp: 2026-05-05T09:38:02.512Z
Learning: When reviewing code that catches `ServiceValidationError` in `*.server.ts` files, do not blindly forward `error.status` to HTTP responses, because SVEs may be thrown with non-default statuses (e.g., 400/500) and forwarding them can cause client-visible behavioral regressions (e.g., surfacing 500s to clients). Prefer a safe default response status of `error.status ?? 422`, but only after confirming via the reachable call graph that the caught `ServiceValidationError` instances are expected to carry those non-default statuses; otherwise, normalize to `422` to avoid unexpected client-visible 5xx behavior.
Applied to files:
apps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/services/metadata/updateMetadata.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2026-05-14T08:21:07.614Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3614
File: apps/webapp/app/v3/mollifier/mollifierGate.server.ts:48-52
Timestamp: 2026-05-14T08:21:07.614Z
Learning: When using Trigger.dev v3 feature flags in the webapp, prefer the existing per-org gating mechanism supported by `flag()` via the `overrides` argument. Pass `Organization.featureFlags` (from `environment.organization.featureFlags`) as the `overrides` value; overrides must take precedence over the global `featureFlag` row. Do not require schema changes or add an `orgId` field to `FlagsOptions` for per-org gating—use the overrides pattern consistently (e.g., in gate flows like `resolveOrgFlag` and any server code that threads `environment.organization.featureFlags` into the gate call).
Applied to files:
apps/webapp/app/v3/runTableV2.server.tsapps/webapp/app/v3/runHierarchy.server.tsapps/webapp/app/v3/runTableV2Status.server.ts
📚 Learning: 2026-03-26T09:02:07.973Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 3274
File: apps/webapp/app/services/runsReplicationService.server.ts:922-924
Timestamp: 2026-03-26T09:02:07.973Z
Learning: When parsing Trigger.dev task run annotations in server-side services, keep `TaskRun.annotations` strictly conforming to the `RunAnnotations` schema from `trigger.dev/core/v3`. If the code already uses `RunAnnotations.safeParse` (e.g., in a `#parseAnnotations` helper), treat that as intentional/necessary for atomic, schema-accurate annotation handling. Do not recommend relaxing the annotation payload schema or using a permissive “passthrough” parse path, since the annotations are expected to be written atomically in one operation and should not contain partial/legacy payloads that would require a looser parser.
Applied to files:
apps/webapp/app/services/metadata/updateMetadata.server.ts
📚 Learning: 2026-05-22T11:50:56.079Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 3333
File: apps/webapp/app/runEngine/services/triggerFailedTask.server.ts:76-80
Timestamp: 2026-05-22T11:50:56.079Z
Learning: When reviewing changes related to org-scoped ClickHouse / org reassignment, treat ClickHouse as the source of truth for the affected read paths (e.g., run lists, span detail, logs) with no Postgres fallback. Reassigning an org from one ClickHouse cluster to another must be done by migrating that org’s existing ClickHouse data between clusters first—do not assume it’s sufficient to update only the OrganizationDataStore entry. If any implementation/change would make org reassignment possible, ensure the required migration design/implementation is included (work tracked under Linear TRI-9659, sub-issue of TRI-7994). During the initial rollout of org-scoped ClickHouse (PR `#3333`), no production org has an override configured, so the limitation is not yet reachable in production; don’t introduce production-dependent assumptions that rely on overrides being active.
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/services/triggerFailedTask.server.ts
🪛 ast-grep (0.44.0)
apps/webapp/test/runTableFkDriftGuard.test.ts
[warning] 54-54: Regular expression constructed from variable input detected. This can lead to Regular Expression Denial of Service (ReDoS) attacks if the variable contains malicious patterns. Use libraries like 'recheck' to validate regex safety or use static patterns.
Context: new RegExp(REFERENCES\\s+(?:"[A-Za-z0-9_]+"\\.)?"${table}", "i")
Note: [CWE-1333] Inefficient Regular Expression Complexity
(regexp-from-variable)
🔇 Additional comments (22)
apps/webapp/app/v3/runHierarchy.server.ts (1)
2-2: LGTM!Also applies to: 18-27, 35-46, 60-67, 70-83
apps/webapp/app/routes/resources.runs.$runParam.ts (1)
10-10: LGTM!Also applies to: 106-110, 120-128, 195-196
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.append.ts (1)
39-40: LGTM!Also applies to: 50-67
apps/webapp/app/routes/realtime.v1.streams.$runId.$target.$streamId.ts (1)
17-33: LGTM!Also applies to: 49-50, 60-71, 180-181, 193-201
apps/webapp/app/routes/api.v1.runs.$runId.spans.$spanId.ts (1)
129-133: LGTM!apps/webapp/app/services/metadata/updateMetadata.server.ts (1)
357-364: LGTM!Also applies to: 379-383
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (1)
1430-1430: LGTM!Also applies to: 1547-1561
internal-packages/run-engine/src/engine/tests/cancelling.test.ts (1)
3-3: LGTM!Also applies to: 231-332, 338-343
apps/webapp/test/runTableFkDriftGuard.test.ts (1)
1-98: LGTM!apps/webapp/test/updateMetadata.test.ts (1)
3-3: LGTM!apps/webapp/app/v3/featureFlags.ts (1)
62-66: LGTM!Also applies to: 98-136
apps/webapp/app/v3/runTableV2.server.ts (1)
3-12: LGTM!Also applies to: 22-63
apps/webapp/app/routes/admin.api.v1.orgs.$organizationId.feature-flags.ts (1)
5-5: LGTM!Also applies to: 88-95
apps/webapp/app/routes/admin.api.v2.orgs.$organizationId.feature-flags.ts (1)
8-13: LGTM!Also applies to: 121-129
apps/webapp/test/featureFlagInvariants.test.ts (1)
1-44: LGTM!internal-packages/run-store/src/types.ts (1)
235-246: LGTM!Also applies to: 347-352, 377-377, 389-389, 400-400
apps/webapp/app/v3/runTableV2Status.server.ts (1)
1-114: LGTM!apps/webapp/test/runTableV2Status.test.ts (1)
1-55: LGTM!apps/webapp/app/runEngine/services/triggerTask.server.ts (1)
28-28: LGTM!Also applies to: 155-167, 721-738
apps/webapp/app/runEngine/services/triggerFailedTask.server.ts (1)
13-14: LGTM!Also applies to: 72-84, 285-303
apps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts (1)
2-2: LGTM!Also applies to: 15-15, 25-36, 153-219, 231-327, 339-339, 398-409, 429-461, 492-500
apps/webapp/test/oneTimeUseTokenClaim.test.ts (1)
1-169: LGTM!
…throwing The backfill cursor format changed from a bare run id to a composite <createdAt>_<id>, and decodeBackfillCursor threw on anything without the separator. A backfill in flight across that change hands the new decoder an old bare-id cursor, so it would throw on every batch. Treat an unparsable cursor (a legacy bare id, or corrupt) as "no cursor" and restart the window: re-backfill is idempotent (ClickHouse ReplacingMergeTree keyed by run id), so the in-flight job self-recovers instead of failing. Logs a warning. Adds cursor round-trip and legacy-format tests.
Reverts 525e363 (graceful legacy-cursor handling). The legacy bare-id cursor only matters for a backfill in flight across the single deploy that changed the cursor format, and that backfill is internal and admin-triggered, so the one-time transition is handled operationally: do not run a backfill during the rollout deploy. Keeping decodeBackfillCursor throwing surfaces genuine cursor corruption loudly instead of silently restarting the window. The old keyset was id-ordered and the new one createdAt-ordered, so a stale cursor cannot be safely resumed anyway, only restarted.
…en the recovery query - updateMetadata cross-table test: wrap the body in try/finally so stopFlushing always runs and the flush loop cannot bleed into later tests on a failure path. - cancelling cross-table cancel-cascade test: poll for the child CANCELED status with a deadline instead of a fixed 1s sleep, to de-flake it under slow CI. - recover-stuck-runs: constrain each UNION branch by id = ANY(runIds) so the recovery query scans only candidate rows instead of unioning both full tables before the join.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New task runs can be routed to a parallel
task_run_v2Postgres table instead of the mainTaskRuntable, decided per-org by a feature flag and keyed purely by the run id's format. Existing runs stay inTaskRun, with no backfill. The flag ships off, so behavior is unchanged until an org is opted in.This builds on the RunStore adapter that already funnels all Postgres
TaskRunaccess through one place (writes in #3981, reads in #3990). RunStore now routes each run to its physical table by id format: a KSUID id meanstask_run_v2, anything else (including legacy cuids) meansTaskRun.Design
runTableV2flag on; everyone else keeps minting legacy ids. The flag is read in memory at the single mint site in the trigger path, so the hot path adds no query. RunStore never sees the flag: it routes purely byisKsuidId(id), and a malformed id falls back to legacy.findRunsdoes a bounded two-way merged keyset cursor (ordered reads standardize on a(createdAt, id)keyset, since cuid and KSUID do not share a sort range), and a non-idfindRun(idempotency-key dedup, or "are there any runs in this environment") queries both tables. Both apply identical scoping to each table, so a merge cannot leak a run across an auth boundary.TaskRun,task_run_v2, and the mollifier buffer, so a reused key is always found and never produces a duplicate run.task_run_v2from the start (empty until orgs cut over), streaming its WAL rows through the same transform into the same ClickHouse table.task_run_v2carries the same relations asTaskRun, and the incoming foreign keys pointing atTaskRunare dropped so the two tables are not coupled by constraints.Stacked on #3990 (its base), so this PR shows only the routing commits on top of the read adapter.
Before enabling the flag for any org,
task_run_v2needsREPLICA IDENTITY FULLapplied the same out-of-band way asTaskRun, so its update and delete events stream to ClickHouse with the old row.