Optional technical reference. This material is for developers, integration specialists and platform administrators. For the consultant lesson, read the practical guide.
Preserved from source baseline
d4587c7c0e073e58c2d430342f65ac55dd701bf4; source/static examples retain their stated scope. They are not new runtime verification or required consultant setup steps.
Current model-key correction — 8 November 2026. The preserved examples below contain historical camelCase record-type/field keys. Do not copy those keys into the current editor or API: use record type leave_request and the lowercase field keys in the current model worksheet, including request_id, employee_id, manager_id, start_date, end_date, requested_days and decision_comment. Form, rule and case input names are separate declarations; map them explicitly rather than mechanically renaming every input. The preserved body remains historical source material, not a current executable model payload.
Data Model Schema Publishing
Build the logical contract before touching a database
A data model describes business record types, fields, identities and relationships. A model draft, a published immutable model version and an APPLIED physical publication are separate states. Saving a field does not create a database table. Publishing a schema version does not prove a target deployment succeeded. Runtime consumers must use the actual applied publication and schema coordinates required by their native contract.
For the leave application, model key leave contains type leaveRequest. Its requestId is a business identifier; the default physical primary key policy remains the model’s system ID unless the builder deliberately selects a supported alternative. Text employee/manager identifiers in this teaching model are data, not authentication or row-authorization grants. Human tasks use actual admitted identity references through their own mapping.
| Layer/operation | Input to retain | What it establishes |
|---|---|---|
| Create/edit draft | Draft ID and sound revision |
An editable logical proposal, not live tables. |
| Validate | Exact schema/target context and issues | Supported schema semantics; no application of DDL. |
| Publish schema version | Draft and required impact coordination | Immutable logical version. |
| DDL/activation preview | Target connector/namespace and current target state | An issued plan to review, not execution. |
| Physical publication | Exact version, target, original idempotency/impact intent | Deployment outcome such as APPLIED, with durable evidence. |
| Runtime consumption | Model/version/type and applicable publication pin | The specific admitted source for queries/actions. |
Prerequisites include a permitted workspace, model authoring role, target connector visibility, and explicit operator authority for any actual DDL. Keep logical validation and physical recovery available as distinct learning paths. See the leave data and rules tutorial for the full handoff to rules and interfaces.
Worked examples
Example DATA-MODEL-01 — Create a modeled leave-request draft with explicit field meaning
Scenario and inputs. Nadia has model-management permission and the required scoped editor role in the actual people-ops workspace. She wants a small new teaching model, not a binding to an existing production table. The body below is the current create-draft request shape with MODELED fields; business sample values are entered later through an admitted action surface.
Procedure. 1. Review each field’s meaning and choose portable types before creating the draft. 2. Send the body to POST /api/data-models/drafts under Nadia’s normal workspace session. 3. Retain the returned actual draft ID and revision, then inspect the saved model in Studio. 4. Validate the schema and any chosen destination constraints before publishing. 5. Confirm requestId uniqueness/business identity and the separate default primary key policy; do not assume a display name or employeeId text creates native user authorization.
{
"key": "leave",
"name": "Leave requests",
"description": "Synthetic teaching application",
"schema": {
"recordTypes": [
{
"key": "leaveRequest",
"name": "Leave request",
"pluralName": "Leave requests",
"origin": "MODELED",
"fields": [
{
"key": "requestId",
"name": "Request ID",
"type": {
"mode": "PORTABLE",
"portable": "STRING"
},
"required": true,
"unique": true
},
{
"key": "employeeId",
"name": "Employee identifier",
"type": {
"mode": "PORTABLE",
"portable": "STRING"
},
"required": true,
"unique": false
},
{
"key": "managerId",
"name": "Manager identifier",
"type": {
"mode": "PORTABLE",
"portable": "STRING"
},
"required": false,
"unique": false
},
{
"key": "startDate",
"name": "Start date",
"type": {
"mode": "PORTABLE",
"portable": "DATE"
},
"required": true,
"unique": false
},
{
"key": "endDate",
"name": "End date",
"type": {
"mode": "PORTABLE",
"portable": "DATE"
},
"required": true,
"unique": false
},
{
"key": "requestedDays",
"name": "Requested days",
"type": {
"mode": "PORTABLE",
"portable": "INTEGER"
},
"required": true,
"unique": false
},
{
"key": "status",
"name": "Status",
"type": {
"mode": "PORTABLE",
"portable": "ENUM",
"enumValues": [
"PENDING",
"APPROVED",
"DECLINED"
]
},
"required": true,
"defaultValue": "PENDING"
},
{
"key": "decisionComment",
"name": "Decision comment",
"type": {
"mode": "PORTABLE",
"portable": "TEXT"
},
"required": false,
"unique": false
}
],
"naturalKey": [
"requestId"
]
}
],
"relationships": [],
"metadata": {}
}
}
Expected result. The request creates an editable logical draft. It defines calendar dates, an integer requested-day quantity and an explicit status enumeration; it does not create a workflow, authenticate Mohammad, deploy DDL or deduct a leave balance.
Failure and recovery. A duplicate machine key, invalid name/type or unsupported target facet must be fixed in the draft and revalidated. Do not switch origin to BOUND or change a shared database merely to get past validation; choose the actual intended storage model and keep source ownership explicit.
Sources. DataModelController.java, RecordType.java, FieldTypeRef.java.
Example DATA-MODEL-02 — Save against the actual revision and publish a logical release
Scenario and inputs. Nadia and another builder opened the same leave-model draft. Nadia’s response contains revision 7, and she changes the description while preserving the full intended schema. The update body uses the real revision concurrency token; the example below is a request-field fragment because the current schema must be included according to the actual edit, not replaced with an empty placeholder.
Procedure. 1. Reload the draft if its current state is uncertain and keep its returned revision. 2. Submit PUT /api/data-models/drafts/{draftId} with the reviewed name/description/schema and expectedRevision: 7. 3. On success, retain the new returned revision and validate the exact saved draft. 4. Review the owning publication-impact path and call /drafts/{draftId}/publish-version with its actual required coordination, including X-Impact-Receipt where applicable. 5. Record the returned immutable version separately from the draft ID/revision.
{
"expectedRevision": 7,
"description": "Leave requests with a reviewed decision comment"
}
Expected result. A matching revision lets the intended edit proceed; publication produces a logical version that subsequent consumers can name. expectedUpdatedAt remains an older compatibility field and is not the preferred sound concurrency token. These numbers belong to separate lifecycles and must not be substituted for one another.
Failure and recovery. If another author has already changed the draft, preserve Nadia’s intended edits, reload and reconcile them with the newer schema. Blindly replacing the expected revision or using a timestamp to force acceptance can overwrite work; a fresh explicit review is required before a new publish attempt.
Sources. DataModelController.java, DataModelController.java.
Example DATA-MODEL-03 — Activate one exact published version on a reviewed destination
Scenario and inputs. The logical leave model now has a reviewed published version, and an authorized operator has admitted a new teaching PostgreSQL destination and namespace. Nadia can view that connector and has the model publication authority. This is a physical-schema operation; the sample names and original intent IDs must be replaced by the operator’s actual issued values.
Procedure. 1. Resolve the exact model/version and connector namespace. 2. Request the supported DDL and activation-impact previews and review their plan, target state and governance facts. 3. Complete the actual impact review so the current issued deployment plan/receipt matches the intended target. 4. Submit POST /api/data-models/{modelId}/versions/{version}/publications with the original operation’s target, idempotency key and receipt. 5. Retain the resulting publication ID/status and inspect it through /api/data-models/publications/{publicationId} before binding consumers.
{
"targetConnectorId": "actual-approved-connector-id",
"targetSchema": "leave_training",
"idempotencyKey": "actual-original-activation-intent",
"impactReceiptId": "actual-issued-impact-receipt"
}
Expected result. An APPLIED result is physical deployment evidence for that exact target; it is different from an immutable model version. The request cannot infer its connector from a label or silently select one of several same-version destinations.
Failure and recovery. A changed target, expired/mismatched impact context, absent connector visibility or uncertain reply requires inspection of the original plan/publication. Do not mint a fresh idempotency key simply because the client timed out; first establish whether the original deployment was admitted and what it did.
Sources. DataModelController.java, DataModelController.java.
Example DATA-MODEL-04 — Treat a partially applied DDL plan as a recovery case
Scenario and inputs. An operator deploys a reviewed model change to a dialect whose DDL cannot be rolled back as one transaction. The publication fails after some physical statements may already have succeeded. Application consumers still need an unambiguous model/publication state; a failed HTTP request is not proof that no schema effect occurred.
Procedure. 1. Preserve the original publication ID, idempotency intent, target and statement-level diagnostics. 2. Inspect the stored publication and safe physical-state evidence using the authorized operator path. 3. Identify which statements are confirmed applied, which failed and which remain uncertain before deciding on any continuation. 4. Compare that state with the current reviewed target plan and follow the dialect-specific supported recovery. 5. Revalidate consumer source coordinates and only advertise the source as usable after its actual publication state supports it.
Expected result. Recovery is based on durable evidence and explicit target state, not a generic retry button or a fresh model version that hides the old failure. Existing data, audit history and other applications’ tables remain under their original ownership.
Failure and recovery. Do not run an automatic down migration, DROP schema or delete publication evidence to manufacture a clean start. If physical custody or outcome evidence is unavailable, retain the accountable operator and request the exact missing state confirmation; continue independent logical-model documentation while that external operation remains unresolved.
Sources. DataModelController.java, DataModelController.java.
Detailed reference
Data model make-live publishes MODELED record types to an external database through a JDBC connector. A connector is eligible only when all of these are true:
protocol=JDBC- metadata or capabilities include
allowSchemaDeploy=true
Connectors may additionally declare schemaDeployNamespaces as an ordered
allowlist. The first entry is the preferred Studio destination. Namespace
discovery marks every other visible schema or database as ineligible, and the
backend enforces the same policy before it reserves a publication, persists a
DDL plan or receipt, or executes target DDL. The bundled hardened PostgreSQL
connector allows and prefers inflow_sync; its JDBC current schema (public)
is intentionally not a deployment target.
- metadata declares a supported
sqlDialect - the backend dialect registry has an implemented schema-deployment dialect for that id
The backend rejects unsupported dialects before planning or executing DDL. Studio uses /api/data-models/deployment-dialects plus the connector list to show only real deployment targets and to explain ineligible connectors.
Business Types And Physical Storage
Studio exposes business data types. Schema publishing projects those business types to safe physical columns for the selected SQL dialect instead of treating the business type as the database type.
| Business type | Platform meaning | Physical projection | Default handling |
|---|---|---|---|
AUTO_NUMBER |
Business-readable generated number, such as AST-0001. Inflow allocates the value. |
Required unique text column (STRING(128), rendered per dialect). |
Plain defaults are rejected; the platform generates values before writing records. |
USER |
Reference to a stable Inflow user identity. | Text identity column (STRING(64), rendered per dialect). |
Plain SQL defaults are rejected; runtime current-user behavior belongs in platform logic. |
DOCUMENT |
Reference to a managed Inflow document. | Text identity column (STRING(64), rendered per dialect). |
Plain SQL defaults are rejected; uploads and document lifecycle stay in Inflow. |
| Portable primitives | Text, number, date/time, boolean, JSON, enum, binary, list. | Dialect-specific mapping from the portable type. | Validated by business type first, then by dialect literal rules. |
This means AUTO_NUMBER is publishable on MySQL and the other implemented
relational dialects even when the database has no matching native sequence
feature. The database stores the final readable number as text; Inflow owns
allocation, formatting, and concurrency.
Supported Dialects
| Dialect id | Connector templates | Namespace handling | Auto-number | JSON/list mapping | Notes |
|---|---|---|---|---|---|
postgresql |
PostgreSQL | CREATE SCHEMA IF NOT EXISTS |
platform-generated text | jsonb, arrays for scalar lists |
Transactional DDL. |
mysql |
MySQL | CREATE DATABASE IF NOT EXISTS |
platform-generated text | json |
Non-transactional DDL. |
mariadb |
MariaDB | CREATE DATABASE IF NOT EXISTS |
platform-generated text | json |
Non-transactional DDL. |
oracle |
Oracle Database | existing user/schema; no automatic schema/user creation | platform-generated text | clob |
Oracle schemas are users. Create the user/schema and grants before publishing. |
sqlserver |
Microsoft SQL Server | creates schema when missing with SCHEMA_ID guard |
platform-generated text | nvarchar(max) |
Uses uniqueidentifier, bit, and datetimeoffset. |
snowflake |
Snowflake | CREATE SCHEMA IF NOT EXISTS in the connected database |
platform-generated text | variant |
Non-transactional DDL. Standard-table primary, unique, and foreign-key constraints are informational. |
MongoDB and Redis templates are not eligible for table-style schema publishing. They remain available for connected-system integrations, health checks, and future non-relational strategies, but Studio must not present them as make-live database targets.
DDL Coverage
The business type registry owns the logical-to-physical projection. The dialect layer owns identifier quoting, namespace creation, system column types, add-column syntax, comments, and portable physical type mappings. The generic DDL service still owns data-model semantics: table names, primary keys, unique constraints, required/nullability, indexes, relationship key columns, local foreign keys, junction tables, and additive diffs.
Publishing is target-aware. Dependent record-type table renames, system-ID/business-key
identity transitions, populated primary-key replacements, and incompatible
field types require an explicit migration strategy and are rejected before DDL
execution. A relationship-free business primary key can move between existing
fields through the empty-table flow described below. A dependency-free modeled record type can be removed only through
the empty-table flow described below. Verified compatible widening of an
unconstrained field is also supported. Inflow widens STRING capacity and DECIMAL precision/scale
when both fractional and integral capacity are preserved on all six SQL
dialects; PostgreSQL, MySQL, MariaDB, and SQL Server also support INTEGER to
LONG. Oracle and Snowflake keep portable integer-range CHECK constraints, so
that widening remains blocked until the constraint can be migrated with the
column. Fields participating in keys, indexes, uniqueness, or explicit
relationship mappings remain blocked because those dependent objects need a
coordinated migration plan. Capacity reductions and incompatible conversions
remain destructive. Reordering the same enum members is metadata-only because
the physical CHECK represents an unordered set. Adding members is a guarded
constraint replacement. Removing members first scans for stored values outside
the desired set and returns an actionable remediation error when any remain.
Apply repeats the preflight, adds the desired CHECK before removing the exact
previous CHECK, and verifies each semantic effect. Snowflake HYBRID tables
remain unsupported because their CHECK contract cannot be altered safely.
New enum CHECK constraints receive a bounded deterministic name on every
engine that accepts named column constraints. MariaDB rejects that inline
syntax and assigns the column name instead. Target inspection therefore
resolves legacy and MariaDB enum constraints only when exactly one catalog
constraint on that table is semantically equivalent to the previously
applied member set; missing or ambiguous matches fail closed and are never
guessed. MariaDB catalog reads filter information_schema.check_constraints
by TABLE_NAME (not only by schema + constraint name) so other tables that
also have an inline CHECK named after the same column — for example many
status enums in a shared lab database — cannot poison resolution.
MariaDB inline CHECKs are not independently droppable; after resolving the one
exact old expression, Inflow re-declares the same frozen column
type/default/nullability without its inline CHECK while the desired named table
constraint is already active. Any additional unrecognized MariaDB CHECK on
the same table makes that rewrite ineligible rather than risking removal of
an external constraint.
Existing-field default changes affect future writes only and use verified
dialect-specific operations on PostgreSQL, MySQL, MariaDB, Oracle, and SQL
Server. Snowflake can drop an existing literal default, but rejects adding or
replacing one because ALTER COLUMN SET DEFAULT accepts sequences rather than
literal values. SQL Server discovers and removes the column's actual default
constraint before installing the deterministic Inflow constraint.
Required-field additions are accepted only when the business projection emits
a physical, non-NULL literal for that target. Runtime-only defaults therefore
cannot masquerade as a backfill. Oracle additionally rejects empty defaults for
STRING, TEXT, and ENUM because Oracle stores the empty string as NULL.
Existing-field nullability changes use dialect-specific ALTER
operations on PostgreSQL, MySQL, MariaDB, Oracle, SQL Server, and Snowflake.
Making a field required first checks the target for NULL rows and returns a
backfill/removal action without freezing a publication when remediation is
needed; relaxing a required field to nullable needs no data preflight. Snowflake
also requires ARRAY, OBJECT, and VARIANT tables to be empty before those columns
can become required. An existing field's unique flag is a
metadata-only change when that same single field is already enforced by the
model's natural key, business primary key, or a unique index. Otherwise,
PostgreSQL, MySQL, MariaDB, Oracle, and SQL Server run a duplicate-data preflight
and add a unique index only when the data is clean; Preview returns actionable
remediation without freezing a publication when duplicates exist. SQL Server's
preflight also treats multiple NULL rows as a conflict because an ordinary
SQL Server unique index permits only one. Snowflake rejects this migration:
standard-table uniqueness is informational and HYBRID table constraints cannot
be added after table creation. Existing uniqueness removal uses the exact-object
resolution and receipted drop flow described below.
Adding a natural key or an explicit unique index to an existing PostgreSQL,
MySQL, MariaDB, Oracle, or SQL Server table uses the same preflight, including
all columns of a composite key. Rows containing NULL in any key column are
ignored where the database's ordinary unique semantics treat them as distinct;
SQL Server checks duplicate tuples including NULL. Snowflake standard-table
additions remain informational constraints and therefore do not claim or run an
enforcement preflight.
A materialized natural key can be replaced or removed through the same frozen
drop effect used for indexes. Replacement preflights the new key tuple, drops
the old named natural-key object, creates the new one, and verifies the final
inventory. Moving a single-column natural key to that field's unique flag is
metadata-only because the existing unique object already provides the same
enforcement. A natural key that is also the business primary key remains a
primary-key migration. Snowflake HYBRID natural keys remain create-time-only.
An existing relationship-free business primary key can be replaced with another
business key on PostgreSQL, MySQL, MariaDB, and SQL Server when the target table
is empty and no field, natural-key, index, or relationship mutation is bundled
into the same version. Preview verifies the current named primary key and checks
that the table is empty without freezing a failed plan. Apply takes a
write-excluding lock, repeats the empty-table check, resolves the exact current
primary-key name, and freezes a target-specific replacement statement plus its
semantic before/after effect. PostgreSQL performs one transactional
ALTER TABLE; MySQL and MariaDB use their atomic DROP PRIMARY KEY, ADD PRIMARY KEY form; SQL Server runs the named drop/add batch under the same transaction
and lock. Receipt recovery adopts an already-completed replacement only after
the desired ordered columns are proved. Populated tables require a user-authored
backfill/deduplication/cutover strategy, and related types require an ordered
foreign-key strategy. Moving between system ID and a business key also changes
record identity and column shape, so it remains blocked pending an explicit
identity-remapping contract. Oracle is blocked because implicit DDL commits
break the preflight lock boundary; Snowflake primary-key changes are blocked
because its enforced HYBRID constraints are create-time-only and standard-table
keys are informational.
An explicit unique index whose ordered columns exactly match the natural or
business primary key is likewise metadata-only; neither duplicate DDL nor a
second inventory expectation is emitted. Removing that suppressed index
definition is also metadata-only because no separate target index ever
existed. A materialized RecordIndex can be removed or replaced through a
frozen dialect-specific drop effect. Unique replacements use the duplicate-data
preflight before the plan is frozen. PostgreSQL and SQL Server transactionally
drop the old named object and create the new definition. MySQL and MariaDB first
create a deterministic temporary unique guard over the desired columns, then
drop the old object, create the final object, and remove the guard. Oracle also
creates the guard first, but after dropping the old object it renames the guard
to the final name because Oracle rejects a second index over an identical column
list. Every intermediate operation is receipted and semantically inspected, so
a failure cannot silently leave the table without either the old or desired
uniqueness protection. MySQL, MariaDB, Oracle, and Snowflake use absence-aware
statement receipts so an accepted operation is never replayed after an
ambiguous connection outcome.
Snowflake standard-table RecordIndex definitions are unique constraints and
use DROP CONSTRAINT; non-unique indexes remain unsupported. Snowflake HYBRID
indexes are create-time-only, so removal and replacement are rejected as a
genuine dialect limitation. Removing inline field uniqueness is supported on
PostgreSQL, MySQL, MariaDB, Oracle, SQL Server, and Snowflake standard tables.
Before freezing the plan, Inflow first prefers the deterministic managed _uq
index used by a prior additive uniqueness change; otherwise it resolves exactly
one engine-catalog UNIQUE constraint with the reviewed column order. Missing or
ambiguous matches fail closed. The resolved object name, dialect-specific drop
SQL, and semantic removal effect are frozen and receipted, so an ambiguous
non-transactional outcome is adopted only after absence is proven. Snowflake
HYBRID uniqueness removal and primary-key changes remain blocked. Relationship
endpoint, cardinality, and mapping changes remain fail-closed until their
dependent transitions are receipted; local delete-action changes follow the
guarded behavior below.
A new nullable field, a new required field with a safe default, a new type, a
new index, and a new relationship remain supported additions.
Removing a dependency-free field is supported on PostgreSQL, MySQL, MariaDB,
and SQL Server only when target inspection proves the physical column
still matches the frozen prior shape and a bounded preflight finds no non-NULL
values. Preview returns remediation without freezing a publication when data
would be lost. Apply repeats the check while holding a write-excluding table
lock on the same connection as the DDL, closing the concurrent-write gap.
Lock acquisition can briefly block application traffic and fails the
publication rather than proceeding from a stale preflight. Fields used by an
auto-number, primary/natural key, unique rule, index, or relationship remain
blocked until those dependencies have their own ordered migration strategy.
A remove/add pair that looks like a rename is never assumed to preserve data:
it succeeds automatically only when the old column is empty; populated
renames require an explicit dual-write/backfill/cutover strategy. Studio field-key
renames now carry an explicit migrationRenamedFrom intent. When the source
column exists, the destination does not, and the field is dependency-free with
an unchanged physical contract, Preview emits one semantic RENAME_COLUMN
effect and Apply preserves every value. Rename plus type, default, nullability,
uniqueness, key, index, or relationship changes must be split into later
publications; Inflow never guesses a rename from matching names or types.
Oracle and Snowflake support DROP COLUMN syntax, but cannot preserve a
write-excluding preflight lock across their implicit DDL commit boundary.
Automatic field removal is therefore rejected for those engines and requires
a quiesced, explicit migration strategy.
Removing a modeled record type is permitted on PostgreSQL, MySQL, MariaDB, and
SQL Server only when it has no modeled relationships, no auto-number state, and
its target table is both empty and semantically identical to the last applied
frozen table shape. Preview performs a read-only shape and row preflight. Apply
takes a write-excluding lock, repeats both checks, drops the table, and verifies
its absence before marking the receipt complete. A populated table returns an
actionable remediation result; dependent types require an ordered dependency
strategy. Oracle and Snowflake are blocked because their implicit DDL commit
boundary cannot preserve that lock through DROP TABLE; they require an
explicit quiesced migration strategy. A modeled record type can change its key
without losing rows when Studio records an explicit migrationRenamedFrom
source and the physical shape is otherwise unchanged. Preview never infers a
rename from similar names or schemas: it proves that the exact source table
exists, the destination does not, and the source still matches the frozen
applied shape. Apply uses a semantic before/after table-rename receipt, so an
ambiguous non-transactional outcome is adopted only when the destination is
exact and the source is absent. PostgreSQL, MySQL, MariaDB, Oracle, SQL Server,
and Snowflake all have a single-table rename operation; MySQL/MariaDB use atomic
RENAME TABLE, SQL Server uses sp_rename, and the other dialects use ALTER TABLE ... RENAME TO. The initial automatic contract is deliberately limited to
MODELED types without relationships, natural keys, named indexes, inline
uniqueness, or auto-number state. Those dependencies derive physical names or
external identities from the type key and require an ordered rename/remapping
strategy. A bundled field, system-column, or primary-key shape change is also
blocked and must be published separately, except for enabling the supported
soft-archive policy. That additive case freezes distinct source and destination
column inventories, renames the populated table first, and then adds the three
lifecycle columns with their safe defaults. Older frozen rename plans that do
not contain a distinct source inventory continue to use their persisted table
shape for both sides.
Adding a new AUTO_NUMBER field to an already-live modeled type is supported
when the target table is empty. Preview checks that prerequisite without
freezing a publication; Apply locks and checks again, creates or repairs the
counter table and bootstrap row, and adds the required unique text column.
The first record allocation initializes the field counter at one. Populated
tables require an explicit backfill strategy so every existing record receives
a stable generated value before the field becomes required.
Changing an existing AUTO_NUMBER field to or from another type remains
destructive. Prefix or padding changes are physically metadata-only, but can
collide with previously generated values. Inflow therefore accepts them only
when the modeled target table is empty. Preview checks that condition without
freezing a publication; Apply locks and rechecks both the modeled table and
counter table, then uses a receipted sentinel transition to insert or reconcile
the counter's prefix/padding without resetting its numeric position. Populated
tables require an explicit counter and collision-remediation strategy.
Before adding an enforced foreign key over existing mapped columns, Preview
checks for orphan tuples and returns a repair action without freezing the plan.
Generated optional relationship columns begin as NULL and need no orphan
scan. A generated required relationship column can be added to an existing
table only when that table is empty; populated owners require row removal, an
optional relationship, or an explicit backfill strategy before publication.
An existing generated relationship can become required after its key columns
are backfilled; Preview checks for NULL, and the frozen plan alters every key
column. It can later become optional by relaxing those columns. An explicitly
mapped relationship may change requiredness only when every mapped owner field
changes its required flag consistently, so the normal field transition chain
provides the physical enforcement.
Removing a local one-to-one or one-to-many relationship is data-preserving:
the frozen plan drops the exact generated foreign key and relationship index,
then makes generated required owner columns nullable. Existing reference values
and columns remain available for audit or a later explicit cleanup. Explicitly
mapped model fields are not altered. Many-to-many junction removal,
cross-database relationship cleanup, relationship remapping, and Snowflake
HYBRID relationship removal remain blocked until they have an explicit artifact
and data-retention strategy.
Changing only the delete action of a local foreign key is also target-aware.
PostgreSQL, MySQL, MariaDB, and SQL Server install a temporary foreign key with
the desired action, drop the canonical constraint, recreate it with the same
stable name, and then remove the guard. This keeps referential enforcement
continuous across transactional and non-transactional execution. Oracle
rejects two foreign keys over the same owner/reference columns (ORA-02275),
while dropping and recreating its constraint crosses implicit DDL commits;
Preview therefore rejects that transition until the user supplies an explicit
quiesced migration strategy. Snowflake standard tables omit delete actions, so
such a model change is physically metadata-only there. Snowflake HYBRID changes
that render to a different action remain blocked because their constraints are
create-time only. Non-local relationships have no target FK and their action
change is metadata-only after normal model validation.
The transactional preflight defers a final constraint-inventory mismatch only
when each missing index or foreign key is supplied by an earlier statement in
the same frozen plan; unrelated target drift still fails closed.
DDL preview and publication use the same target-aware planner. For each
connector and target schema, planning compares the requested version with that
target's latest APPLIED publication. A target that is already live therefore
receives an additive diff in Preview rather than a misleading full-create plan.
Column mutation effects record both the reviewed prior shape and the desired shape. Receipt-backed, non-transactional engines execute an ALTER only while the target still matches that frozen prior shape; unrelated drift fails closed. Effect specification version 2 introduced this precondition. An unresolved older frozen plan requires explicit reconciliation rather than automatic replay.
Non-transactional DDL Recovery
MySQL, MariaDB, Oracle, and Snowflake can commit individual DDL statements even when a later statement fails. For those dialects, Inflow freezes the generated plan in platform PostgreSQL, fingerprints every semantic statement, and creates a durable receipt before target execution. Each statement is inspected before and after execution. An exact effect from a previously started attempt is verified without replay; a conflicting or unprovable effect fails closed.
Receipts use fenced platform leases. A worker that loses its holder/fence token
cannot change a receipt or mark the publication applied. APPLIED requires all
receipts to be verified and a final semantic inspection of the complete plan.
Target connections and statements are prepared while the receipt is still
PENDING, so a preparation failure is known not to have invoked target DDL and
can be retried automatically. Once execute has been invoked, an ABSENT or
UNKNOWN catalog result never authorizes replay, regardless of elapsed time or
the number of observations. The receipt remains AMBIGUOUS until the exact
effect becomes visible or an operator reconciles the target explicitly. A
PostgreSQL transition trigger enforces the same fail-closed state machine during
rolling deployments, including against older application nodes. Existing shared
namespaces and explicit managed ENSURE_* effects may be adopted only when their
inspected semantics are exact; an unreceipted exact business table is treated as
a collision.
Persisted plans created before semantic effect specifications are not replayed automatically. Operators must reconcile those targets explicitly. Live every- statement recovery certification for all non-transactional engines remains a release gate; this mechanism must not be interpreted as completed engine certification until that evidence is recorded in the certification matrix.
Retrying a publication that is already APPLIED is always read-only. Inflow
semantically inspects the operational ENSURE_* effects and returns the
original publication only when every effect is exact. A missing, conflicting,
or unverifiable operational prerequisite returns
DATA_MODEL_OPERATIONAL_REPAIR_REQUIRED; the retry executes no repair DDL,
does not change the original status or appliedAt, and does not repeat the
publication audit or Fabric registration. Until a separately identified,
fenced, receipted, and audited repair operation is available, operators must
repair the prerequisite through an explicit controlled database change and
then retry for read-only verification.
Failure Diagnostics And Secret Safety
Database and provider error messages are treated as untrusted secret-bearing input. Publication rows, publication audit metadata, REST responses, Studio, and Fabric registration warnings never retain or render a JDBC/SQL/provider message. They expose only these bounded machine contracts:
DATA_MODEL_PUBLICATION_FAILEDwithDatabase schema publication failed.FABRIC_REGISTRATION_WARNwithThe database schema was applied, but Fabric registration failed.DATA_MODEL_OPERATIONAL_REPAIR_REQUIREDwith the fixed read-only repair instruction.
A safe summary may append a generated UUID diagnostic id. The same id is
recorded in the failure audit and warning log so an operator can correlate the
incident without copying SQL, identifiers, JDBC URLs, usernames, or passwords.
Logs record only the exception class and bounded driver codes such as SQLState
and numeric vendor code; they never record the exception message or stack trace.
Fabric registration failure has its own
DATA_MODEL_FABRIC_REGISTRATION_FAILED audit event. A control-plane migration
rewrites legacy publication error text to the fixed summaries and installs a
database constraint that rejects any future nonconforming value.
Datasource resolution happens inside the fenced publication failure boundary.
If it fails after the PLANNED row is created, the row is terminally changed to
FAILED with the same safe diagnostic contract; it is not left indefinitely
PLANNED. Operational verification remains read-only and returns only the safe
repair-required response plus its diagnostic id.
Snowflake standard tables do not implement ordinary secondary indexes.
Explicit non-unique RecordIndex definitions are therefore rejected before a
Snowflake plan can emit DDL; they are never silently dropped. Natural keys,
unique RecordIndex definitions, and one-to-one relationship keys are rendered
as named Snowflake UNIQUE constraints so the published metadata remains
truthful, although Snowflake does not enforce those constraints on standard
tables. Non-unique relationship lookup indexes are an internal optimization and
are omitted on Snowflake; correctness must not depend on them.
Durable EVENT Webhook Synchronization
EVENT-mode BOUND types receive signed webhook events into replicas hosted in
the platform PostgreSQL database. An event with a nonblank eventId is claimed
in data_model_webhook_event_inbox as a SHA-256 fingerprint, scoped by
workspace, model, and record type. Business record keys are fingerprinted too;
the inbox stores no webhook payload or connector secret. The claim, one replica upsert or soft delete, the stored row-count
outcome, and DATA_MODEL_WEBHOOK_EVENT_APPLIED audit evidence use the same
platform transaction.
If replica DML, audit persistence, or outcome persistence fails, PostgreSQL
rolls back the replica change and the inbox claim. Redelivery therefore remains
retryable, including after an API restart. Once that transaction commits, a
duplicate delivery is a no-op and produces neither another replica mutation nor
another audit event. Events without an eventId preserve legacy processing
behavior and are transactional, but cannot be deduplicated; production webhook
senders should always supply a stable event id.
Successful non-capped scheduled or operator-triggered BOUND-type sync runs emit
DATA_MODEL_SYNC_SUCCEEDED audit evidence with the run id, sync mode,
upsert/delete counts, duration, and resulting replica size. Capped and failed
runs emit DATA_MODEL_SYNC_CAPPED and DATA_MODEL_SYNC_FAILED respectively.
Terminal run persistence, terminal state persistence, and terminal audit
evidence share one platform transaction in the Spring-wired service. Snapshot,
incremental, and CDC replica writes still require the broader durable
coordinator work for cursor receipts, commit ambiguity, and restart recovery.
CDC (Debezium) resume state in v1 is source-managed, not platform-managed:
the adapter persists offsets through a node-local FileOffsetBackingStore
file, ignores the persisted platform cursor on start, and reports empty
per-event offsets. The platform cursor therefore records only an explicit
offsetStore=source-managed marker — never a fabricated position. Restart
resume works on the node that owns the offset file; relocating a CDC stream
to another node requires sharing or reseeding that file (distributed CDC
offset storage is deferred).
Replica Record Keys, Watermarks, And Reconcile Bounds
A replica row's _record_key is the row's natural-key values joined with
|, with no escaping. The format is persisted in every replica table and is
deliberately frozen: natural-key values containing | can collide with
another key tuple, so natural keys on synced types must not contain |.
Incremental watermarks for timestamp-typed fields persist in a canonical, offset-bearing ISO-8601 instant form and resume by re-binding that instant as a timestamp parameter, so neither the JVM default zone nor the source database session zone can shift the comparison. Watermarks persisted by older builds (JVM-zone text) still resume through the legacy cast path and are rewritten in canonical form on the next run. Sources that sort NULL first (SQL Server) page the watermark column with an explicit NULLS LAST ordering so a leading run of NULL-watermark rows cannot stall the cursor.
RECONCILE delete detection sweeps the source key set in keyset pages but
holds the full source and live-replica key sets in memory to diff them. Both
the data-model and the Fabric replication engines bound the two sides by
the sync policy's maxRows cap: a source-size preflight rejects an oversized
source as CAPPED before the first page is pulled, so the worst case is twice
the configured cap. A merge-style streaming diff was considered and deferred
as a structural change.
Operator Checklist
- Create or connect a JDBC system from a supported relational template.
- Keep
allowSchemaDeploy=trueonly for databases where Inflow is allowed to create objects. - Ensure the database principal can create the target namespace when the dialect supports it.
- For Oracle, pre-create the target user/schema and grant table, index, constraint, and sequence privileges.
- Use business types for user-facing concepts. Do not model generated numbers, people, or managed documents as plain strings just to satisfy one database.
- Use the DDL preview endpoint before publishing high-risk models.
- Configure EVENT webhook senders to retry failures and reuse the same stable
eventIdfor every delivery attempt.