GgenIgniter.Reactors.ReconcileReactor (ggen_igniter v26.9.8)

Copy Markdown View Source

The real Reactor coordination pipeline for ggen_igniter's reconciliation spine: observe -> load -> resolve -> query -> render -> admit -> actuate -> verify -> finalize evidence. use Reactor (plain Reactor, NOT Ash.Reactor -- ggen_igniter must stay usable without Ash as a mandatory runtime dependency).

Corrections applied (2026-08-27), on top of the design below

Two real architectural corrections from the user, applied directly to this module (this section documents WHAT changed and WHY; the rest of this moduledoc, describing the pipeline's shape, is otherwise unchanged):

A. A receipt for every admitted attempt, not only success

Quoting the user directly:

If files were actually changed -- even temporarily -- then a consequential physical actuation occurred... the run receipt should record ACTUATION_STARTED -> files A,B changed -> verification failed -> compensation started -> A,B restored -> resulting project hash == pre-run hash -> standing = COMPENSATED.

Before this correction, :receipt only ran on the success path (gated behind :commit_manifest's own success, itself gated behind :verify) -- a refusal in :admit, a compile failure in :verify, or a partial- actuation self-heal in :actuate produced NO receipt at all, even though real bytes may have hit disk and been reverted. Fixed by:

  • run/1 (this module's new, recommended public entry point -- Reactor.run(__MODULE__, %{reconcile_opts: opts}) still works, but only run/1 guarantees a persisted receipt on every path) wraps the bare Reactor.run/4 call and, on ANY {:error, _} result, derives a real standing_for_failure/2 from which step failed and why, then persists a real GgenIgniter.Receipt via GgenIgniter.Receipt.append!/2 before returning.
  • :actuate (both its internal self-heal branch in actuate_pending/2 AND its real Reactor undo/3, triggered when :verify fails afterward) emits real GgenIgniter.Telemetry.OcelEmitter events (ACTUATION_STARTED / FILES_CHANGED / COMPENSATION_STARTED / FILES_RESTORED, the last carrying real pre_run_hash/ post_run_hash/matches_pre_run_hash computed via GgenIgniter.Receipt.hash_entries/1 and hash_files/1). :verify's own compile failure is tagged reason_type: "build_broken" specifically -- a real, distinct meaning from a generic verification failure, giving standing_for_failure/2 a genuine basis for :build_broken vs. the more general :compensated.
  • The four standings (:alive / :refused / :compensated / :build_broken) are GgenIgniter.Receipt.standing/0's own closed set, not invented here.

See test/ggen_igniter_receipt_compensated_test.exs for the real, running proof: a genuinely invalid Elixir template makes :verify's real mix compile --warnings-as-errors fail for real, Reactor's real undo/3 restores the real pre-run file content, and the persisted receipt's pre_run_hash == post_run_hash is asserted from the real file on disk -- no mock anywhere in that chain.

B. Evidence finalization is one boundary, not two independent steps

Quoting the user directly:

verify succeeds -> manifest advances -> receipt write fails. Now you have standing state with no standing evidence.

The former :commit_manifest + :receipt two-step split is replaced by ONE :finalize_evidence step (finalize_evidence/1) that, in this exact order:

  1. Prepares BOTH the next manifest content (commit_recipe/5, reusing GgenIgniter.Manifest's own real API, unchanged) and the new GgenIgniter.Receipt payload, entirely in memory -- nothing durable written yet.
  2. Persists the receipt FIRST: GgenIgniter.Receipt.append!/2, a real append-only File.write!/3. If this itself raises, :finalize_evidence fails like any other step -- since :actuate already wrote real files, Reactor's own undo/3 rolls them back (a real :compensated outcome, recorded by run/1's failure path, since this step never gets to build a receipt of its own in that case).
  3. Only once that append genuinely succeeds does it attempt to promote the manifest, via GgenIgniter.Manifest.persist!/2's OWN existing temp-file-then-File.rename!/2 atomic-rename protocol (unchanged; not reimplemented here). If THIS specific call fails, it is caught LOCALLY (not re-raised) -- the attempt is still genuinely :alive (files were written and verified correctly; rolling them back because the manifest CACHE failed to update would be wrong) -- metadata["manifest_promotion"] instead records the pending state, with the now-durable receipt as the real recovery anchor for a retry to reconcile against.

This ordering makes "manifest advanced but no receipt exists to explain why" structurally impossible: the manifest write is not even ATTEMPTED until the receipt line is already flushed to disk. See test/ggen_igniter_finalize_evidence_ordering_test.exs for a real, no-mock proof: the manifest's own target path is pre-created as a DIRECTORY (so File.rename!/2 genuinely raises File.RenameError, not a simulated failure), and the test asserts the real receipt file already contains the real :alive line while the manifest path is still untouched.


Grounded directly in real prior art explored this session (~/ash_r2rml's reactor_pipeline.ex + Admission module, ~/xaas's Actuation.Reactor, ~/ex4pm's "Reactor is the single workflow execution kernel" rule, ~/ggen's sync.rs plan/actuate split) -- see ~/.claude/plans/i-want-you-to-humming-knuth.md sections 1-2 for the full design rationale. This module implements sections 1-2, PLUS (per the corrections above) a real, working slice of sections 3-4 (a real GgenIgniter.Receipt and real GgenIgniter.Telemetry.OcelEmitter event emission) -- not the full SHA-256 hash-CHAIN across receipts those sections may still describe, which remains legitimate future work.

Steps

observe_prior_manifest  -- pure read: GgenIgniter.Manifest.load/1
load_ontology           -- pure read: GgenIgniter.Ontology.load!/1
resolve_pack            -- pure read: GgenIgniter.Pack.resolve_dir!/1
run_queries             -- GgenIgniter.Engine.fetch!/run, per target
render                  -- GgenIgniter.Render.render/2 PLUS
                            GgenIgniter.Manifest lookups, produces the
                            real intended delta as `[%PendingActuation{}]`
                            (one create/replace/eval intent per target,
                            plus one real `:delete` item per stale-prune
                            candidate) -- never a bare `{out_path,
                            content}` pair; nothing is written yet (the
                            same plan/actuate split as ggen's own
                            `sync.rs` PendingWrite/SyncReport -- see
                            GgenIgniter.PendingActuation's own moduledoc)
admit                   -- accumulating, fail-closed gate inspecting the
                            FULL `[%PendingActuation{}]` plan (not just
                            output paths): real duplicate-target
                            refusal, real refusal of any `:delete` item
                            lacking `ownership: true`, and
                            GgenIgniter.Manifest's real stale/--on-stale
                            policy -- never reimplemented
actuate                 -- the ONLY step that touches the filesystem for
                            real create/replace/eval intents
                            (GgenIgniter.Actuate.write_file!/3,
                            eval_code!/2), consuming each
                            `%PendingActuation{}`'s own `operation`
                            field directly rather than re-deriving it;
                            tracks exactly what it wrote, in its own
                            return value, for real undo. Real `:delete`
                            items pass through admitted but unactuated
                            here -- see "Prune timing" below
verify                  -- a real `mix compile --warnings-as-errors`
                            subprocess against the actuated project.
                            A compile failure here is the real
                            `:build_broken` standing (correction A).
finalize_evidence       -- receipt persisted FIRST, manifest promoted
                            atomically only after (correction B); also
                            where real --on-stale prune deletions run
                            (see "Prune timing" below), only after
                            :verify

return :finalize_evidence

:observe_prior_manifest, :load_ontology, and :resolve_pack all depend only on the :reconcile_opts input, so Reactor's own dependency-graph scheduler runs them concurrently -- no manual concurrency management for that part of the pipeline.

Multi-target input shape

:reconcile_opts accepts the exact same flat opts GgenIgniter.Reconcile.run/1 does (:ontology/:query/:template/:pack/:pack_dir/:engine/:mode/ :out/:unless_exists/:skip_if/:dry_run, plus :on_stale/:manifest_dir/ :verify_cwd, new here) for the single-output case -- this is what makes test 1's byte-for-byte parity against Reconcile.run/1 meaningful: with no :targets key, this pipeline runs exactly one target, built from the flat opts themselves.

Passing :targets (a list of keyword lists, each a per-target override of :query/:template/:mode/:out/:unless_exists/:skip_if/:dry_run, merged onto the shared top-level opts) runs N independent render/actuate targets in one Reactor pipeline run -- this is the real, in-memory "PendingWrite list" the plan's :render step describes, and is what the concurrency proof in the test suite exercises. :ontology/:engine/:pack/ :pack_dir at the TOP level are shared across every target (loaded/resolved once); a target may still override its own :pack/:pack_dir/:engine for its own template/query resolution.

v26.9.2 (workstream B): :for_each_row is one more real per-target override key -- a string-keyed row map (one row of a --for-each driver query's results), merged into that target's own EEx bindings LAST by run_target_queries/3 (same precedence Mix.Tasks.GgenIgniter.Sync.build_bindings/2 documents). This is how Mix.Tasks.GgenIgniter.Sync.run_for_each_via_reactor!/7 expands ONE --for-each NAME-bearing request into N real targets sharing this same :targets mechanism, rather than inventing a second, parallel fan-out path inside this module.

Same-output-path collision: real, explicit refusal (never last-writer-wins)

:actuate runs independent targets' real writes concurrently (Task.async_stream/3) for real throughput on independent files. Two DIFFERENT targets resolving to the SAME real output location in the SAME run would otherwise race -- there is no principled dependency order the ontology itself expresses for two unrelated queries that happen to render the same path. :admit detects this structurally (grouping this run's own pending file-mode writes by each item's real canonical_target -- GgenIgniter.ArtifactIdentity.canonicalize/2's result, NEVER the raw target string) and refuses the ENTIRE run with {:error, {:refused_duplicate_output_path, [...]}} before any actuation happens at all -- chosen deliberately over inventing an implicit dependency-ordering mechanism the ontology has no way to express; see the test suite's concurrency proof for the real, asserted behavior.

Correction (2026-08-27): grouping by raw string was a real, confirmed gap

.ggen_igniter_factory/redteam-concurrency-nondeterminism.md (an independent adversarial review, its real reproducer re-run against this fix in test/ggen_igniter_artifact_identity_test.exs) found that this guard used to group by the raw target STRING (Enum.group_by(& &1.target)) -- so two targets whose --out/to: strings differed only by a redundant /./ segment (the same root cause covers //, ..-traversal, and symlink-based aliases) resolved to the SAME real inode while comparing as different Elixir strings, silently bypassing this guard entirely. :actuate's real Task.async_stream/3 then genuinely raced both writes against the identical real file -- confirmed, empirically, as real nondeterministic last-writer-wins (both possible targets independently observed as the real surviving winner across repeated real runs), with the pipeline reporting standing: :alive (full, false success) regardless of which target's content was actually discarded. Fixed by grouping on GgenIgniter.PendingActuation's real canonical_target field instead -- see GgenIgniter.ArtifactIdentity for the real canonicalization primitive this and within_root?/2's real path-escape guard (also enforced in :admit, alongside this check) are built on.

compensate/4 vs undo/4 -- real Reactor semantics, not folklore

Reactor's real Reactor.Step behaviour (confirmed by reading deps/reactor/lib/reactor/step.ex and deps/reactor/documentation/tutorials/02-error-handling.md directly, not assumed) gives each step TWO distinct rollback hooks with different triggers:

  • compensate/4 fires when THIS step's OWN run/3 returns {:error, reason} -- it decides whether to retry/continue/fail, it does not by itself revert a DIFFERENT, already-successful step.
  • undo/4 fires when a LATER step in the same reactor run fails, and Reactor needs to roll back THIS already-successful step's work.

The scenario this module's key test proves -- :verify (a later step) fails after :actuate has genuinely written files -- is exactly undo/4's real trigger, not compensate/4's. :actuate implements BOTH, honestly: undo/4 is the real, tested revert mechanism for "a later step failed"; compensate/4 handles :actuate's OWN run failing (e.g. one target's write raising mid-loop) by self-healing INSIDE run/3 before ever returning {:error, ...} (reverting every write this SAME invocation already made), so compensate/4 itself has nothing left to do and correctly returns :ok.

:verify scope

Runs a real mix compile --warnings-as-errors subprocess (System.cmd/3) against reconcile_opts[:verify_cwd] || reconcile_opts[:manifest_dir] || File.cwd!() -- the actuated project's own directory. mix format --check-formatted-equivalent verification (the plan's "in-process where feasible" aside) is deliberately deferred this pass -- the compile check is the load-bearing proof this pipeline's compensation exists to protect against, and is what the task's own adversarial test exercises.

Prune timing: deliberately AFTER :verify, not before

Mix.Tasks.GgenIgniter.Sync applies --on-stale prune's real deletions right after its own writes, with no compile-check gate at all. This pipeline moves real prune deletions into :commit_manifest -- i.e. only after :verify has confirmed the newly-actuated project actually compiles -- a strictly safer ordering (never delete a stale path until the new state is confirmed good), disclosed here as a deliberate difference from sync.ex's existing behavior, not an oversight.

Testing hooks (inert in real use)

Three optional, per-target opts keys exist ONLY to make :actuate's real concurrency, and its compensation path, independently observable from a test, and are otherwise inert no-ops:

  • :test_delay_ms -- Process.sleep/1 for this many ms immediately before this target's real write.
  • :test_probe -- an atom naming a public ETS table; if given, {{index, :start}, monotonic_ms} and {{index, :stop}, monotonic_ms} are inserted around the (possibly delayed) write, so a test can assert two targets' real write windows actually overlapped.
  • :test_chmod_after_write -- an integer POSIX mode (e.g. 0o444); if given, File.chmod!/2 is called with this mode against the target path IMMEDIATELY after this target's real write genuinely succeeds (deterministic, not timing-dependent: :actuate is a separate, strictly-later Reactor step than :verify/:actuate's own undo, so the file is guaranteed read-only well before either ever runs). This is what lets a test construct a REAL "compensation itself fails" scenario without racing a background process against the pipeline: the real mutation genuinely happens first (this target's write succeeds normally), then the file becomes genuinely unwritable, then a later step's real failure triggers a real revert attempt against it -- see "Compensation failure: :compensation_failed" below and test/ggen_igniter_compensation_failure_test.exs.

None of the three is read anywhere outside actuate_one/2 below, and all are nil (no-op) unless a caller deliberately sets them.

Compensation failure: :compensation_failed (the sixth -- catastrophic -- outcome)

Every rollback attempt this module makes (revert_all/1, called from both :actuate's real undo/3 -- a LATER step failed -- and from actuate_pending/2's own internal self-heal -- :actuate's OWN run failed) used to call revert_one/2 directly and let a real revert failure (a target that became read-only or was deleted out from under the process) RAISE uncaught. Reactor's own do_undo/6 (deps/reactor/lib/reactor/executor/step_runner.ex) has NO rescue around its call to Step.undo/4 -- confirmed by reading that file directly, not assumed -- so a raise from inside undo/3 used to propagate as a bare, uncaught exception all the way out of Reactor.run/4, past this module's OWN run/1 case entirely. That is strictly WORSE than mis-reporting :refused: no receipt was persisted at all, for an attempt that genuinely mutated disk and then failed to restore it -- exactly the silent-catastrophe class this module's evidence guarantee (correction A) exists to close.

Fixed: revert_all/1 is now a real, best-effort, NEVER-RAISING operation -- each path's own revert is individually rescued (revert_one_safe/2), so one path's failure does not stop attempts on the others, and the overall result is a tagged {:ok, restored_paths} or {:error, %{paths:, restored:, failed:}} (failed :: [{path, {module, message}}]). Both call sites (:actuate's undo/3 and actuate_pending/2's self-heal branch) now inspect this tagged result: on :error, they emit a real COMPENSATION_FAILED OCEL event and return {:error, {:compensation_failed, details}} from their OWN run/undo-callback -- Reactor's documented, non-raising failure contract ({:error, reason}), never a raw exception. run/1's error branch (find_compensation_failure/1) recursively searches the returned Reactor.Error class for this specific tagged reason -- regardless of which step name Reactor happens to attach it to (:actuate's undo failing surfaces as an UndoStepError; :actuate's own self-heal failing surfaces as a RunStepError for :actuate itself) -- and when found, builds a GgenIgniter.Receipt with standing: :compensation_failed whose reason names, explicitly and in one sentence: that a real mutation occurred, that verification failed, that restoration (compensation) itself ALSO failed, the exact paths that could not be restored and why, which paths (if any) WERE restored, and that manual repair may be required. See GgenIgniter.Receipt's moduledoc for this standing's full contract, and test/ggen_igniter_compensation_failure_test.exs for the real, no-mock proof (File.chmod!/2 on a real target makes a real revert write fail, via the :test_chmod_after_write hook above).

Telemetry

middlewares do middleware Reactor.Middleware.Telemetry end wires Reactor's own built-in middleware, emitting real :telemetry.execute/3 events for this reactor's run/step/compensate/undo timing under [:reactor, :run, :start | :stop], [:reactor, :step, :run, :start | :stop], [:reactor, :step, :guard, :start | :stop], [:reactor, :step, :process, :start | :stop], [:reactor, :step, :compensate, :start | :stop], and [:reactor, :step, :undo, :start | :stop] -- see test/ggen_igniter_reconcile_reactor_telemetry_test.exs for the real, no-mock proof (a real :telemetry.attach_many/4 handler process receiving real events from a real ReconcileReactor.run/1 execution).

Alongside Reactor.Middleware.Telemetry, GgenIgniter.Reactors.CompensationTelemetryMiddleware is wired here too -- it turns three of the same real Reactor lifecycle events ({:compensate_start, _} / {:compensate_error, _} / :undo_start) plus two real error/2-derived standings (:compensation_failed / :build_broken, via the same find_compensation_failure/1/ find_step_error/2 helpers run/1 itself uses) into durable, real ETS counters (CompensationTelemetryMiddleware.counters/0) instead of ephemeral :telemetry events -- see that module's own moduledoc for the full real-event-shape citations and test/ggen_igniter_reconcile_reactor_compensation_telemetry_test.exs for the real, no-mock proof.

Summary

Functions

Read-only admission preview backing mix ggen_igniter.plan (Mix.Tasks.GgenIgniter.Plan -- see that module's moduledoc for the full CLI contract). Runs the SAME observe-prior-manifest -> load-ontology -> resolve-pack -> run-queries -> render -> admit sequence run/1 runs for a real reconciliation -- reusing the exact same private helpers (normalize_targets/1, run_target_queries/3, build_plan/3, admit_pending/2) rather than a parallel plan-only implementation -- but STOPS before :actuate/:verify/:finalize_evidence ever run: nothing is written to disk, no GgenIgniter.Receipt is persisted, and no manifest is promoted. This is a plain function, not a Reactor.run/4 invocation -- there is nothing here for Reactor's compensation/undo machinery to protect against, since no mutation ever happens on this path.

The recommended entry point: runs one full reconcile attempt for reconcile_opts (the same flat/:targets shape this module's steps accept -- see moduledoc). Unlike calling Reactor.run(__MODULE__, %{reconcile_opts: reconcile_opts}) directly, this function GUARANTEES a real GgenIgniter.Receipt is persisted (GgenIgniter.Receipt.append!/2) on every path: :alive on success (already true of the bare :finalize_evidence step), and :refused / :compensated / :build_broken on the three real failure paths, which :finalize_evidence structurally cannot reach itself (it only runs after :admit + :actuate + :verify all succeed). See moduledoc's "Corrections applied" section (correction A).

Functions

plan(reconcile_opts)

@spec plan(keyword()) ::
  {:ok, [GgenIgniter.PendingActuation.t()]}
  | {:error, {:unsupported_capability, String.t()}}
  | {:error, term()}

Read-only admission preview backing mix ggen_igniter.plan (Mix.Tasks.GgenIgniter.Plan -- see that module's moduledoc for the full CLI contract). Runs the SAME observe-prior-manifest -> load-ontology -> resolve-pack -> run-queries -> render -> admit sequence run/1 runs for a real reconciliation -- reusing the exact same private helpers (normalize_targets/1, run_target_queries/3, build_plan/3, admit_pending/2) rather than a parallel plan-only implementation -- but STOPS before :actuate/:verify/:finalize_evidence ever run: nothing is written to disk, no GgenIgniter.Receipt is persisted, and no manifest is promoted. This is a plain function, not a Reactor.run/4 invocation -- there is nothing here for Reactor's compensation/undo machinery to protect against, since no mutation ever happens on this path.

Returns {:ok, [%PendingActuation{}]} -- the exact admitted plan :admit would hand to :actuate, unwrapped from admit_pending/2's own %{pending: pending, ...} map since callers of plan/1 (today, only Mix.Tasks.GgenIgniter.Plan.report/3) only need the list itself.

{:error, {:unsupported_capability, reason}} when the resolved template has a --- frontmatter header -- this bounded pipeline (like run/1, via Mix.Tasks.GgenIgniter.Sync's own run_via_reactor/3 guard) does not implement frontmatter parsing. {:error, reason} for any other admission-time refusal (one of admit_pending/2's own tagged reasons: :refused_duplicate_output_path / :refused_path_escapes_root / :refused_unowned_delete / :refused_stale_outputs).

Raises ArgumentError for an unresolvable input (missing/ambiguous template, ontology, or query) -- the same vocabulary resolve_ontology_path!/1 and resolve_template_path!/1 already raise for run/1; Mix.Tasks.GgenIgniter.Plan rescues this itself and turns it into exit code 2, this function does not catch its own raises.

run(reconcile_opts)

@spec run(keyword()) ::
  {:ok, GgenIgniter.Receipt.t()} | {:error, GgenIgniter.Receipt.t()}

The recommended entry point: runs one full reconcile attempt for reconcile_opts (the same flat/:targets shape this module's steps accept -- see moduledoc). Unlike calling Reactor.run(__MODULE__, %{reconcile_opts: reconcile_opts}) directly, this function GUARANTEES a real GgenIgniter.Receipt is persisted (GgenIgniter.Receipt.append!/2) on every path: :alive on success (already true of the bare :finalize_evidence step), and :refused / :compensated / :build_broken on the three real failure paths, which :finalize_evidence structurally cannot reach itself (it only runs after :admit + :actuate + :verify all succeed). See moduledoc's "Corrections applied" section (correction A).