Inflow.aiDocs

Optional technical reference. This material is for developers, integration specialists and platform administrators. For the consultant lesson, read the practical guide.

Preserved from source baseline d4587c7c0e073e58c2d430342f65ac55dd701bf4; source/static examples retain their stated scope. They are not new runtime verification or required consultant setup steps.

Durable Runtime State

Follow a waiting execution across a restart

This guide explains why an actual workflow can continue on another runtime node and what that durability does—and does not—promise. The source model is pinned published BPMN; the execution state is the current instance, waiting/active tokens, variables, work items, timers and events. A business reference such as leave-2026-014 is useful to people, but engine continuation also needs the actual instance/token/node correlation.

Runtime consistency map

Durable item Why the next node needs it Diagnostic question
Instance key/version/status Select original model and legal operation Is this the actual instance, workspace and release?
Waiting work item/token Correlate the exact next advancement Was this token already consumed?
Variable values Rehydrate the case file Which declared values were committed?
Timers/events Preserve due work and ordering Is this current work or retained history?
Parent/child links and deferred arrivals Respect pause and case structure Was arrival retained pending activation?
Transaction manager Commit or roll back local changes together Is this the JDBC runtime’s actual atomic boundary?

Read the current snapshot and permitted history before proposing a recovery. SQL tables listed below describe implementation contracts, not permission to edit rows. Transactional token claiming is internal to continuation. Service-task/external/message/agent/timer queues have separate leases; the retired HTTP token renew/release endpoints are not administrative recovery tools.

Most importantly, a database rollback cannot retract a remote request already sent to another system. Explain local atomicity separately from remote side effects. The timelines below are source-backed mechanics examples; they are not newly executed crash, database or active-active acceptance tests.

Worked examples

Example FLOW-DR-01 — Continue waiting work on a replacement node

Scenario and inputs. A leave case reaches Review on runtime nodeA. Its instance, published definition/version, variables and waiting work are persisted before nodeA becomes unavailable. An authorized nodeB later handles the actual task completion. The user-facing business reference is known, but the supported completion path still supplies the server-owned task/work-item correlation.

Procedure. Read the persisted instance and its exact published model coordinates. Load durable variables and the current waiting work from the runtime state repository. The wired restorable engine rehydrates that selected model/state before the supported resume/signal/cancel operation executes. Continue through the admitted task or worker API rather than starting another case. Inspect the resulting token, work item and instance snapshot after acceptance.

Expected result. NodeB can continue the original execution even though nodeA created it. The case retains the original source version and committed variables, and the waiting work is consumed as part of normal advancement.

Failure and recovery. If the published model or correlation evidence is absent, report the actual failure and preserve the stored state. Do not rebuild the case from a current draft, guess a token by node label, or start a duplicate request to make the UI appear responsive.

Sources. WorkflowRuntimeServiceImpl.java:1247; CrashRecoveryRuntimeTest.java:1.

Example FLOW-DR-02 — Understand why the second continuation is refused

Scenario and inputs. Two deliveries refer to the same waiting Review work item. They carry the same actual instance/token/node correlation through the supported runtime path. The first delivery has already advanced the token. This is different from two separate legitimate human responses required by an authored shared-completion policy.

Procedure. Inspect the first accepted completion and the token’s resulting state. A subsequent attempt to resume that consumed token is judged against current durable state. Read the actual task receipt/run result and next snapshot before deciding what recovery is necessary. Keep business response reconciliation at the task or worker boundary; a low-level token refusal is not an invitation to manufacture another token.

Expected result. The atomic claim/consume guard prevents the same ACTIVE/WAITING token from being advanced twice. Once consumed, that original token is no longer resumable and can produce RUNTIME_TOKEN_NOT_RESUMABLE. Its lock ownership is cleared as part of consumption.

Failure and recovery. Do not copy a newly observed token into the old command, retry a different node ID, or start a fresh case to evade the refusal. If the first response was lost, use the canonical accepted-operation evidence available at the owning boundary rather than treating the second refusal as proof nothing happened.

Sources. WorkflowRuntimeServiceImpl.java:1272.

Example FLOW-DR-03 — Separate local rollback from a remote side effect

Scenario and inputs. A JDBC-backed workflow claims a waiting token and then encounters an engine failure before local advancement can commit. The deployment uses the actual Spring transaction-manager wiring. A previous external step may also have contacted another system, so there are two distinct consistency questions: local case state and remote business effect.

Procedure. Inspect the safe incident/history and committed instance state. The runtime transaction covers token/work-item changes, engine execution, variable persistence, timers/events and instance update. If that transaction failed, determine which local changes rolled back before choosing the supported continuation/recovery. Separately consult the owning integration/run evidence to establish whether a remote request was rejected, accepted or remains unknown.

Expected result. Local token consumption and related JDBC writes do not become a half-committed workflow advancement. A rolled-back local claim can leave the original waiting work available for an admitted recovery. This is a local transactional guarantee, not a distributed exactly-once guarantee.

Failure and recovery. Never assume the remote system rolled back merely because PostgreSQL did. If external outcome is unknown, reconcile it before another consequential dispatch. Preserve failure evidence rather than deleting an incident or directly changing token status to make the database look successful.

Sources. WorkflowRuntimeServiceImpl.java:4760.

Example FLOW-DR-04 — Retain a signal while the parent is suspended

Scenario and inputs. An authorized operator suspends a leave case before a matching signal arrives. The signal has the expected name and valid declared payload. The case’s parent/child state and current workspace scope matter; a display label is not a signal correlation key or permission grant.

Procedure. Send/observe the signal through its existing admitted API. While the case is paused, inspect its current state and retained deferred arrival rather than trying to force immediate engine progression. Activate the parent through the supported operator operation with its reason, then read the resulting snapshot/events. If a child had been paused independently, inspect it separately rather than assuming every child was resumed.

Expected result. The valid arrival survives the pause and is replayed through the normal declared-output/variable checks on activation. Duplicate pending deliveries for the same waiting location do not create repeated advancement.

Failure and recovery. An invalid payload is refused before it can be treated as an accepted deferred command. A replay failure retains actionable evidence and its supported correction path. Do not call retired token lease routes, manually delete a queue row, or equate activation with permission to repeat an uncertain external action.

Sources. LifecycleCaseRuntimeTest.java:91; WorkflowRuntimeController.java:163.

Detailed reference

The runtime persists the execution-state layer needed for active-active workflow execution, including instances, tokens, variables, work items, timers, and engine events.

Persisted State

flowchart TD
    A["WorkflowRuntimeService"] --> B["runtime_process_instances"]
    A --> C["runtime_execution_tokens"]
    A --> D["runtime_variables"]
    A --> E["runtime_work_items"]
    A --> F["runtime_timer_jobs"]
    A --> G["runtime_engine_events"]

runtime_work_items stores the durable task projection, including the authored form key and pinned form version used by Debug and restart recovery.

runtime_execution_tokens stores active, waiting, consumed, cancelled, and failed token records. Tokens include the process instance, current BPMN node, status, timestamps, version, and lock owner fields used for worker-safe claiming.

runtime_variables stores process variables as individual JSON values by process instance and variable name. The process instance table still keeps a snapshot in variables_json, but this table gives the engine a durable variable source for rehydration.

Rehydration

Before resume, signal, or cancel, the runtime checks whether the engine supports RestorableWorkflowEngineService.

If supported, the runtime reloads:

  • BPMN process model by definition key/version
  • persisted variables from RuntimeExecutionStateRepository
  • process instance status

Then it restores the engine before calling resume, signal, or cancel.

When the runtime is constructed with a Spring PlatformTransactionManager, the state-changing runtime operations run inside a single database transaction. This means token consumption, work item completion, engine execution, variable persistence, work item persistence, timer persistence, event persistence, and process instance updates commit together or roll back together.

sequenceDiagram
    participant A as "Node A"
    participant DB as "Database"
    participant B as "Node B"

    A->>DB: Start process and persist waiting token/variables
    B->>DB: Load process instance
    B->>DB: Load BPMN model
    B->>DB: Load variables
    B->>B: Restore engine state
    B->>DB: Begin transaction
    B->>DB: Atomically consume old active/waiting token
    B->>B: Resume engine
    B->>DB: Persist next token/work item/variables
    B->>DB: Commit transaction

Resume Safety

The runtime claims the requested token before completing the related work item or calling the engine. A token claim writes locked_by and locked_until; a different owner cannot consume the token while that lease is still valid, and another owner can reclaim it after expiry. Resume claims and consumes the token in the same transaction (consuming clears locked_by and locked_until), so the claim is an internal resume guard: no production path leaves a token lease held.

The HTTP routes that renewed or released a token lease (POST /api/process-instances/{instanceId}/tokens/{tokenId}/renew-lease and /release-lease) were retired by SCRUM-3764, because they could never find a held lease; they now answer 404. WorkflowRuntimeService.renewTokenLease(...) and releaseTokenLease(...) remain only as deprecated methods, due for removal. The PROCESS_INSTANCE_LEASE_MANAGE permission is kept so stored API-key scopes and role grants that name it still validate, but no route checks it.

The repository only claims or consumes tokens that are currently ACTIVE or WAITING. If the same resume command is replayed, the second call is rejected with RUNTIME_TOKEN_NOT_RESUMABLE because the token is already CONSUMED.

If the engine fails after the token has been consumed, the transaction rolls the token and work item changes back. Another worker can safely retry the same resume later.

Runtime boundary

A fresh runtime node can resume a process started by another engine instance when it has the work-item or timer correlation values.

The runtime now has durable rehydration, token-level lease ownership, atomic duplicate-resume protection, and transactional rollback protection for JDBC deployments wired with a transaction manager.

Token claiming is an internal guard within the resume transaction. The deprecated Java token-lease methods do not make the retired HTTP routes available. External task, message, agent, and timer queues retain their separate worker lease contracts.

Try “document generation”, “integrations” or “first form”.