Skip to main content

Prefect remote run

infrahub-sync ships an optional Prefect integration that exposes one sync run as a Prefect deployment, so a plan or a sync can be started and observed over Prefect's own REST API instead of by shelling out to the CLI.

The flow parameters, returned result fields, and summary log line form the remote-run contract. Changes may extend that contract, but must not silently reshape existing fields.

Trusted development environment only

The default self-hosted Prefect server has no authentication. Run it bound to localhost, on a machine and against an Infrahub instance you are willing to experiment with. It must never be exposed to the public internet.

For a complete, runnable walkthrough — install, load the schema, serve, invoke, inspect, clean up — follow examples/prefect_remote_run/README.md in the repository.

Installation​

Prefect is an optional extra, pinned to a single version:

[project.optional-dependencies]
prefect = ["prefect==3.8.1"]

No published release carries this extra yet, so install it from a repository checkout:

pip install -e '.[prefect]'

Once a release ships the integration, the ordinary extra syntax (pip install 'infrahub-sync[prefect]') applies.

Nothing in the base installation imports, starts, or contacts Prefect. Only the infrahub_sync.orchestration modules import it, and no other part of the package imports those — so ordinary CLI use is unchanged whether the extra is installed or not.

Base dependency change this required​

Making the extra installable changed two base dependency declarations:

  • diffsync[redis]>=2.1,<3.0 became diffsync>=2.1,<3.0 — the [redis] extra caps redis<5.0, which cannot be satisfied next to Prefect's own dependency chain, where pydocket requires redis>=5.
  • redis>=4.3,<9 is now declared directly, because infrahub_sync/utils.py imports RedisStore from DiffSync unconditionally and therefore needs a Redis client in every installation; the floor stays permissive on purpose so anything else requiring diffsync[redis] still resolves.

If you are upgrading an existing installation: the Redis client is now a direct dependency of infrahub-sync with a wider allowed range than before. Redis itself is still only contacted when a sync configuration opts into the Redis store.

Serving the deployment​

The serve process reads the directory holding your sync configurations from the environment and refuses to start without it:

VariableMeaning
INFRAHUB_SYNC_CONFIG_DIRECTORYDirectory containing the sync configurations exposed remotely. Required. A remote caller can only run a configuration found here.
PREFECT_API_URLThe Prefect server to serve against, for example http://127.0.0.1:4200/api.
INFRAHUB_ADDRESS, INFRAHUB_API_TOKENInfrahub credentials, read from the serving process's environment.
export INFRAHUB_SYNC_CONFIG_DIRECTORY="/srv/sync-configs"
python -m infrahub_sync.orchestration.serve

This registers a locally served deployment named run under the flow infrahub-sync, so remote callers look it up at GET /api/deployments/name/infrahub-sync/run. There is no work pool and no separate worker.

Point INFRAHUB_SYNC_CONFIG_DIRECTORY only at configurations you intend to expose: it is the allow-list for remote runs. Relative paths inside a configuration resolve against the serving process's working directory, so start the process from the directory those paths were written for.

Flow parameters​

The flow accepts exactly four parameters. None of them accepts a path, a CLI fragment, a credential, or an environment override.

ParameterTypeDefaultMeaning
sync_namestrrequiredLogical name of a configuration in INFRAHUB_SYNC_CONFIG_DIRECTORY, matched by exact string equality.
operation"plan" or "sync""plan"plan is read-only and maps to the CLI diff lifecycle; sync writes.
confirm_writesboolfalseMust be true for operation="sync". Has no effect on a plan.
branchstr or nullnullInfrahub branch, forwarded exactly as the CLI --branch option. Used only when the configuration's own settings.branch is unset — a configured branch takes precedence over this parameter. With neither set, main is used.

Credentials and endpoints stay in the runner's environment. They are never accepted as parameters and never appear in a returned result.

Failure messages are additionally redacted by value: every configured credential value found in a message — or anywhere in its cause chain — is replaced with ***. Redaction has one deliberate limit: only collected values of six characters or more are replaced, because replacing a shorter value can corrupt ordinary message text (within 6***.0 seconds). A credential shorter than that — a lab-grade CISCO_APIC_PASSWORD=admin — is therefore not redacted from a failure message. Use credentials of realistic length on any runner whose logs are not private.

The confirm-writes gate​

operation="sync" without confirm_writes=true fails before either adapter is loaded and before anything is read or written: the flow run ends FAILED with a state message explaining that confirm_writes=true is required. The same gate applies to any programmatic caller of the shared execution surface, not only to remote runs.

An operation value other than plan or sync is rejected by Prefect's parameter validation when the run is created — the API returns 409 and no flow run is created, so there is no run, no result, and no log output to inspect.

The result​

A successful run returns exactly these fields:

FieldTypeMeaning
sync_namestrResolved logical configuration name
operation"plan" or "sync"Requested operation
run_idstrSync cache run identifier, YYYYMMDDTHHMM-<8 hex>
status"planned", "applied", or "no-change"Terminal outcome
changedboolWhether the run materialized any plan rows
summarydictPer-action counts; create, update, and delete are always all present
artifact_pathstrAbsolute path of the run directory on the runner host

changed is true exactly when status is not no-change, and exactly when the summary counts sum to more than zero. status="planned" occurs only for a plan and status="applied" only for a sync. artifact_path holds the ordinary sync artifacts (run.json, plan.parquet); those files are local to the runner and are not retrievable through Prefect.

changed is not "the destination was written"

changed reports what the run materialized as plan rows, not whether the destination was written. The two can disagree in one case: the engine gates a sync on a recursive difference check, while the plan materializes only the diff root's direct children. A difference that exists solely in nested child elements therefore lets the sync run — and write — while materializing zero rows, so the result comes back status="no-change", changed=false, and an all-zero summary.

Do not read changed=false as "the destination was not touched". For a definitive account of what a run did, read the artifacts under artifact_path on the runner host. This is a known limit of the direct Prefect deployment.

Reading the result remotely​

The flow logs one summary line per run, in a fixed key=value format. That line is the supported way to read a run's outcome remotely — retrieve it with POST /api/logs/filter filtered on the flow-run id:

run 20260731T1058-07e1e25e finished: status=planned changed=True summary=create:5,update:0,delete:0 artifact=/path/to/.infrahub-sync-cache/custom-example/20260731T1058-07e1e25e

Its fields mirror the result: the leading value is run_id, followed by status, changed, the three summary counts, and artifact_path. The format is contractual for this integration and safe to parse. Everything the sync itself logs — load, diff, plan, and per-adapter lifecycle lines — is forwarded into the same flow-run log at INFO, with the originating logger name preserved.

Status mapping​

Prefect statestatusrun.json statusMeaning
COMPLETEDplanneddry-runPlan with changes; nothing was written
COMPLETEDno-changedry-run (plan) or applied (sync)The run materialized no plan rows — normally because the destination already matches the source, but see the changed warning above for the nested-difference case in which a sync writes and still reports no-change
COMPLETEDappliedappliedSync that wrote the changes
FAILEDno result returnedfailed, or no run directory when the refusal precedes executionValidation refusal or execution failure; the sanitized cause is the Prefect state message

Scope and limitations​

The direct deployment does not provide a Sync-owned HTTP API, remote reviewed-plan apply, per-stage tasks, work pools, workers, triggers, or an overlap policy. Saved-plan review and apply remain available through the CLI. Use the separate managed Sync HTTP API when an automation client needs a stable Sync-owned API, durable results and artifacts, reviewed-plan apply, actor authorization, or managed worker execution.

Concurrency guarantees for this direct deployment are limited to the per-configuration lock on one runner host. A second run of the same configuration waits for that lock and fails if it times out. The deployment adds no queue or ordering policy beyond that lock.