Cache layout

infrahub-sync diff and infrahub-sync apply persist run state under:

.infrahub-sync-cache/<sync-name>/
├── .lock                              # per-pipeline filelock (held during runs)
├── last-successful-rowcounts.json     # baseline for the rowcount guardrail
└── <run_id>/
    ├── A/                              # source snapshot
    │   ├── BuiltinTag.parquet
    │   └── ...
    ├── B/                              # destination snapshot
    │   └── ...
    ├── plan/                           # the saved plan artifact — what `apply` reads
    │   ├── operations.jsonl            # one canonical JSON operation per line
    │   └── manifest.json               # the artifact header and its checksum
    ├── plan.parquet                    # the diff plan, for querying (not read by `apply`)
    ├── errors.parquet                  # only when errors > 0
    ├── cursors.json                    # {A: {Resource: cursor}, B: {Resource: cursor}}
    ├── schema-sub-hash.txt             # invalidates the cache when shape changes
    └── run.json                        # status, mode, summary, finished_at

Override the root with INFRAHUB_SYNC_CACHE_DIR=/path/to/shared/cache.

The plan artifact (plan/)

diff and sync both write <run_dir>/plan/, and it is the plan infrahub-sync apply executes — the operations, and the manifest that binds them to a run, a configuration and a set of source snapshots. It holds two files.

It is not the only input to apply. Verification re-reads the source snapshots under A/ that the manifest names, and the command also needs the sync configuration and access to the destination. Archiving plan/ on its own therefore leaves a plan that refuses to apply: keep the whole run directory.

operations.jsonl

One planned operation per line, each a canonical JSON object, every line LF-terminated including the last. Operations are ordered by (tier, operation_id) and apply executes them in exactly that stored order — nothing is re-compared or re-derived at apply time.

Field	Description
operation_id	Stable identifier derived from (action, kind, identity). Unique within a plan; a collision fails the plan run before anything is written.
action	create, update, or delete. An action outside this set is refused while reading, before any destination write.
kind	Destination kind name.
identity	The canonical destination identity — attribute name to value, key-sorted. A value that is itself a reference is a nested {"peer_kind": …, "identity": …} pair.
tier	The write tier the kind belongs to.
payload	The mapped field values to write. Omitted on a delete, which carries none.
relationships	Peer references, each {field, peer_kind, cardinality, peers}, naming peers by kind and identity rather than by a destination-assigned id. Omitted when the operation carries none.

A plan with zero operations writes a present, zero-byte file, so an empty plan is never mistaken for a missing one.

manifest.json

Field	Description
format_version	The artifact format. 2 today. An unsupported format is refused rather than partially read.
run_id	The cache run this plan belongs to.
created_at	UTC ISO-8601 timestamp of the write.
config_version	The configuration the plan was derived from. apply compares it for equality and refuses on a mismatch.
source_snapshot	One {path, digest, row_count} record per source snapshot file the plan was computed against.
operations_count	Number of lines in operations.jsonl. A disagreement makes the artifact torn.
delete_operations_computed	Whether deletes could be enumerated at all — see Deletes in a plan.
destination_binding	The destination the plan was computed against, as {url, branch}: the resolved endpoint URL — environment variables already applied over settings, normalized — and the branch. Never credentials, raw query text, or fragments. Query-bearing endpoints retain a one-way query fingerprint so distinct endpoints still compare differently; fragments are not part of an HTTP destination. apply compares it against the destination it is about to write to and refuses before any write on a mismatch, so a plan reviewed for staging is not applied to production by accident; --allow-destination-change applies anyway. Plans written before this field existed carry no binding, and the comparison is skipped for them.
plan_checksum	Checksum over the manifest body and the operations bytes. run_id, created_at and plan_checksum itself are excluded from it, so two runs of the same configuration over the same source produce the same value.

Every field above is written and read by infrahub-sync itself and is inside the checksum. Manifest fields beyond them are tolerated and preserved — also inside the checksum — so hand-editing one invalidates the plan; an operation record with an unknown field is refused outright.

Write order and the commit point

operations.jsonl is written first and manifest.json last, each atomically through a temporary file and a rename, so neither is ever observed half-written. The manifest's presence is the commit point: a plan/ directory with no manifest is a torn artifact and is refused, while a run directory with no plan/ at all has no plan to read — usually a run predating this format, but equally one whose plan was never written or has since been removed. The reader tells those two verdicts apart by construction rather than by guessing, and the second names all three of its causes.

Deletes in a plan

Deletes are recorded in the plan and never executed against the destination. Applying a delete-bearing plan applies every non-delete operation, deletes nothing, completes as applied, and records the skipped identifiers and their count on the run. That is a designed limitation, not a failure.

Deletes are also only computed when the destination side ran a full extract — an incrementally-loaded destination cannot enumerate what is missing from it. delete_operations_computed records which of the two happened, so a plan whose delete class was never computed is never mistaken for a plan that genuinely has no deletes. Both review depths of infrahub-sync diff --from-plan say so in words.

plan.parquet

Retained for ad-hoc querying, and still written by every diff and sync. It is no longer the input to apply — that is plan/ above. One row per change. The columns are:

Column	Description
action	create, update, or delete. Empty for no-op elements (which are skipped during serialization).
resource	Kind name as declared in schema_mapping[].name.
source_id	DiffSync unique_id of the source-side element.
dest_id	Reserved for the destination's primary key once adapters return it. Empty today.
attribute	Reserved for per-attribute granularity. Empty today (rows are per-element).
old_value	JSON-encoded mapping of {attr: prior_value} from element.get_attrs_diffs()["-"]. Populated on update actions.
new_value	JSON-encoded mapping of {attr: new_value} from element.get_attrs_diffs()["+"]. Populated on create and update.
owner	Reserved for sync-identity-based skip logic. Empty today.
skip_reason	Empty unless the engine deliberately skipped a row.
conflict_class	Empty unless the engine flagged a write conflict.

Query with DuckDB without any import step:

duckdb -c "SELECT action, resource, source_id, new_value FROM read_parquet('.infrahub-sync-cache/from-netbox/<run_id>/plan.parquet') WHERE action <> 'create' LIMIT 20"

Commands

• infrahub-sync diff --name X — writes side A, side B, plan/ and plan.parquet.
• infrahub-sync diff --name X --from-plan <id> — reads back the plan/ of a stored run and writes nothing at all: no adapter is constructed, nothing is extracted, no lock is taken and no run directory is created. See Reviewing and applying a cached plan.
• infrahub-sync sync --name X — runs diff then sync; writes the same cache artifacts as diff plus updates last-successful-rowcounts.json on success.
• infrahub-sync apply --name X --run-id <id> — applies the stored plan/ artifact against the destination without re-extracting the source. Before the first write it verifies the artifact as one gate — run binding, plan checksum, source snapshot, configuration version, and whether the destination adapter implements the planned-write surface — and refuses if any of those checks fails. See Refusals you will meet. The gate does not compare schema-sub-hash.txt: that file guards incremental extraction, not apply. Separately from that gate, apply also refuses when the live destination endpoint or branch differs from the one recorded in the plan, unless --allow-destination-change is passed — which is why the comparison sits outside the gate, whose checks cannot be overridden.
• --allow-rowcount-drop (on sync) bypasses the rowcount guardrail when the operator knows the source has legitimately shrunk.
• --continue-on-error (on sync) skips peer relationships missing identifier values rather than aborting; the engine logs each skip so you can review what was dropped.
• --no-concurrent-load (on diff and sync) falls back to loading source then destination sequentially. The default (concurrent) is safe with all built-in adapters and roughly halves load wall-clock time on real APIs.