The real Reactor coordination pipeline for ggen_igniter's reconciliation
spine: observe -> load -> resolve -> query -> render -> admit -> actuate ->
verify -> finalize evidence. use Reactor (plain Reactor, NOT
Ash.Reactor -- ggen_igniter must stay usable without Ash as a mandatory
runtime dependency).
Corrections applied (2026-08-27), on top of the design below
Two real architectural corrections from the user, applied directly to this module (this section documents WHAT changed and WHY; the rest of this moduledoc, describing the pipeline's shape, is otherwise unchanged):
A. A receipt for every admitted attempt, not only success
Quoting the user directly:
If files were actually changed -- even temporarily -- then a consequential physical actuation occurred... the run receipt should record ACTUATION_STARTED -> files A,B changed -> verification failed -> compensation started -> A,B restored -> resulting project hash == pre-run hash -> standing = COMPENSATED.
Before this correction, :receipt only ran on the success path (gated
behind :commit_manifest's own success, itself gated behind :verify) --
a refusal in :admit, a compile failure in :verify, or a partial-
actuation self-heal in :actuate produced NO receipt at all, even though
real bytes may have hit disk and been reverted. Fixed by:
run/1(this module's new, recommended public entry point --Reactor.run(__MODULE__, %{reconcile_opts: opts})still works, but onlyrun/1guarantees a persisted receipt on every path) wraps the bareReactor.run/4call and, on ANY{:error, _}result, derives a realstanding_for_failure/2from which step failed and why, then persists a realGgenIgniter.ReceiptviaGgenIgniter.Receipt.append!/2before returning.:actuate(both its internal self-heal branch inactuate_pending/2AND its real Reactorundo/3, triggered when:verifyfails afterward) emits realGgenIgniter.Telemetry.OcelEmitterevents (ACTUATION_STARTED/FILES_CHANGED/COMPENSATION_STARTED/FILES_RESTORED, the last carrying realpre_run_hash/post_run_hash/matches_pre_run_hashcomputed viaGgenIgniter.Receipt.hash_entries/1andhash_files/1).:verify's own compile failure is taggedreason_type: "build_broken"specifically -- a real, distinct meaning from a generic verification failure, givingstanding_for_failure/2a genuine basis for:build_brokenvs. the more general:compensated.- The four standings (
:alive/:refused/:compensated/:build_broken) areGgenIgniter.Receipt.standing/0's own closed set, not invented here.
See test/ggen_igniter_receipt_compensated_test.exs for the real, running
proof: a genuinely invalid Elixir template makes :verify's real mix compile --warnings-as-errors fail for real, Reactor's real undo/3
restores the real pre-run file content, and the persisted receipt's
pre_run_hash == post_run_hash is asserted from the real file on disk --
no mock anywhere in that chain.
B. Evidence finalization is one boundary, not two independent steps
Quoting the user directly:
verify succeeds -> manifest advances -> receipt write fails. Now you have standing state with no standing evidence.
The former :commit_manifest + :receipt two-step split is replaced by
ONE :finalize_evidence step (finalize_evidence/1) that, in this exact
order:
- Prepares BOTH the next manifest content (
commit_recipe/5, reusingGgenIgniter.Manifest's own real API, unchanged) and the newGgenIgniter.Receiptpayload, entirely in memory -- nothing durable written yet. - Persists the receipt FIRST:
GgenIgniter.Receipt.append!/2, a real append-onlyFile.write!/3. If this itself raises,:finalize_evidencefails like any other step -- since:actuatealready wrote real files, Reactor's ownundo/3rolls them back (a real:compensatedoutcome, recorded byrun/1's failure path, since this step never gets to build a receipt of its own in that case). - Only once that append genuinely succeeds does it attempt to promote
the manifest, via
GgenIgniter.Manifest.persist!/2's OWN existing temp-file-then-File.rename!/2atomic-rename protocol (unchanged; not reimplemented here). If THIS specific call fails, it is caught LOCALLY (not re-raised) -- the attempt is still genuinely:alive(files were written and verified correctly; rolling them back because the manifest CACHE failed to update would be wrong) --metadata["manifest_promotion"]instead records the pending state, with the now-durable receipt as the real recovery anchor for a retry to reconcile against.
This ordering makes "manifest advanced but no receipt exists to explain
why" structurally impossible: the manifest write is not even ATTEMPTED
until the receipt line is already flushed to disk. See
test/ggen_igniter_finalize_evidence_ordering_test.exs for a real,
no-mock proof: the manifest's own target path is pre-created as a
DIRECTORY (so File.rename!/2 genuinely raises File.RenameError, not a
simulated failure), and the test asserts the real receipt file already
contains the real :alive line while the manifest path is still
untouched.
Grounded directly in real prior art explored this session (~/ash_r2rml's
reactor_pipeline.ex + Admission module, ~/xaas's Actuation.Reactor,
~/ex4pm's "Reactor is the single workflow execution kernel" rule, ~/ggen's
sync.rs plan/actuate split) -- see
~/.claude/plans/i-want-you-to-humming-knuth.md sections 1-2 for the full
design rationale. This module implements sections 1-2, PLUS (per the
corrections above) a real, working slice of sections 3-4 (a real
GgenIgniter.Receipt and real GgenIgniter.Telemetry.OcelEmitter event
emission) -- not the full SHA-256 hash-CHAIN across receipts those
sections may still describe, which remains legitimate future work.
Steps
observe_prior_manifest -- pure read: GgenIgniter.Manifest.load/1
load_ontology -- pure read: GgenIgniter.Ontology.load!/1
resolve_pack -- pure read: GgenIgniter.Pack.resolve_dir!/1
run_queries -- GgenIgniter.Engine.fetch!/run, per target
render -- GgenIgniter.Render.render/2 PLUS
GgenIgniter.Manifest lookups, produces the
real intended delta as `[%PendingActuation{}]`
(one create/replace/eval intent per target,
plus one real `:delete` item per stale-prune
candidate) -- never a bare `{out_path,
content}` pair; nothing is written yet (the
same plan/actuate split as ggen's own
`sync.rs` PendingWrite/SyncReport -- see
GgenIgniter.PendingActuation's own moduledoc)
admit -- accumulating, fail-closed gate inspecting the
FULL `[%PendingActuation{}]` plan (not just
output paths): real duplicate-target
refusal, real refusal of any `:delete` item
lacking `ownership: true`, and
GgenIgniter.Manifest's real stale/--on-stale
policy -- never reimplemented
actuate -- the ONLY step that touches the filesystem for
real create/replace/eval intents
(GgenIgniter.Actuate.write_file!/3,
eval_code!/2), consuming each
`%PendingActuation{}`'s own `operation`
field directly rather than re-deriving it;
tracks exactly what it wrote, in its own
return value, for real undo. Real `:delete`
items pass through admitted but unactuated
here -- see "Prune timing" below
verify -- a real `mix compile --warnings-as-errors`
subprocess against the actuated project.
A compile failure here is the real
`:build_broken` standing (correction A).
finalize_evidence -- receipt persisted FIRST, manifest promoted
atomically only after (correction B); also
where real --on-stale prune deletions run
(see "Prune timing" below), only after
:verify
return :finalize_evidence:observe_prior_manifest, :load_ontology, and :resolve_pack all depend
only on the :reconcile_opts input, so Reactor's own dependency-graph
scheduler runs them concurrently -- no manual concurrency management for
that part of the pipeline.
Multi-target input shape
:reconcile_opts accepts the exact same flat opts GgenIgniter.Reconcile.run/1
does (:ontology/:query/:template/:pack/:pack_dir/:engine/:mode/
:out/:unless_exists/:skip_if/:dry_run, plus :on_stale/:manifest_dir/
:verify_cwd, new here) for the single-output case -- this is what makes
test 1's byte-for-byte parity against Reconcile.run/1 meaningful: with no
:targets key, this pipeline runs exactly one target, built from the flat
opts themselves.
Passing :targets (a list of keyword lists, each a per-target override of
:query/:template/:mode/:out/:unless_exists/:skip_if/:dry_run,
merged onto the shared top-level opts) runs N independent render/actuate
targets in one Reactor pipeline run -- this is the real, in-memory
"PendingWrite list" the plan's :render step describes, and is what the
concurrency proof in the test suite exercises. :ontology/:engine/:pack/
:pack_dir at the TOP level are shared across every target (loaded/resolved
once); a target may still override its own :pack/:pack_dir/:engine for
its own template/query resolution.
v26.9.2 (workstream B): :for_each_row is one more real per-target
override key -- a string-keyed row map (one row of a --for-each driver
query's results), merged into that target's own EEx bindings LAST by
run_target_queries/3 (same precedence
Mix.Tasks.GgenIgniter.Sync.build_bindings/2 documents). This is how
Mix.Tasks.GgenIgniter.Sync.run_for_each_via_reactor!/7 expands ONE
--for-each NAME-bearing request into N real targets sharing this same
:targets mechanism, rather than inventing a second, parallel fan-out
path inside this module.
Same-output-path collision: real, explicit refusal (never last-writer-wins)
:actuate runs independent targets' real writes concurrently
(Task.async_stream/3) for real throughput on independent files. Two
DIFFERENT targets resolving to the SAME real output location in the SAME
run would otherwise race -- there is no principled dependency order the
ontology itself expresses for two unrelated queries that happen to render
the same path. :admit detects this structurally (grouping this run's own
pending file-mode writes by each item's real canonical_target --
GgenIgniter.ArtifactIdentity.canonicalize/2's result, NEVER the raw
target string) and refuses the ENTIRE run with {:error, {:refused_duplicate_output_path, [...]}} before any actuation happens at
all -- chosen deliberately over inventing an implicit dependency-ordering
mechanism the ontology has no way to express; see the test suite's
concurrency proof for the real, asserted behavior.
Correction (2026-08-27): grouping by raw string was a real, confirmed gap
.ggen_igniter_factory/redteam-concurrency-nondeterminism.md (an
independent adversarial review, its real reproducer re-run against this
fix in test/ggen_igniter_artifact_identity_test.exs) found that this
guard used to group by the raw target STRING
(Enum.group_by(& &1.target)) -- so two targets whose --out/to:
strings differed only by a redundant /./ segment (the same root cause
covers //, ..-traversal, and symlink-based aliases) resolved to the
SAME real inode while comparing as different Elixir strings, silently
bypassing this guard entirely. :actuate's real Task.async_stream/3
then genuinely raced both writes against the identical real file --
confirmed, empirically, as real nondeterministic last-writer-wins (both
possible targets independently observed as the real surviving winner
across repeated real runs), with the pipeline reporting standing: :alive (full, false success) regardless of which target's content was
actually discarded. Fixed by grouping on GgenIgniter.PendingActuation's
real canonical_target field instead -- see GgenIgniter.ArtifactIdentity
for the real canonicalization primitive this and within_root?/2's
real path-escape guard (also enforced in :admit, alongside this check)
are built on.
compensate/4 vs undo/4 -- real Reactor semantics, not folklore
Reactor's real Reactor.Step behaviour (confirmed by reading
deps/reactor/lib/reactor/step.ex and
deps/reactor/documentation/tutorials/02-error-handling.md directly, not
assumed) gives each step TWO distinct rollback hooks with different
triggers:
compensate/4fires when THIS step's OWNrun/3returns{:error, reason}-- it decides whether to retry/continue/fail, it does not by itself revert a DIFFERENT, already-successful step.undo/4fires when a LATER step in the same reactor run fails, and Reactor needs to roll back THIS already-successful step's work.
The scenario this module's key test proves -- :verify (a later step)
fails after :actuate has genuinely written files -- is exactly undo/4's
real trigger, not compensate/4's. :actuate implements BOTH, honestly:
undo/4 is the real, tested revert mechanism for "a later step failed";
compensate/4 handles :actuate's OWN run failing (e.g. one target's
write raising mid-loop) by self-healing INSIDE run/3 before ever
returning {:error, ...} (reverting every write this SAME invocation
already made), so compensate/4 itself has nothing left to do and
correctly returns :ok.
:verify scope
Runs a real mix compile --warnings-as-errors subprocess (System.cmd/3)
against reconcile_opts[:verify_cwd] || reconcile_opts[:manifest_dir] || File.cwd!() -- the actuated project's own directory. mix format --check-formatted-equivalent verification (the plan's "in-process where
feasible" aside) is deliberately deferred this pass -- the compile check is
the load-bearing proof this pipeline's compensation exists to protect
against, and is what the task's own adversarial test exercises.
Prune timing: deliberately AFTER :verify, not before
Mix.Tasks.GgenIgniter.Sync applies --on-stale prune's real deletions
right after its own writes, with no compile-check gate at all. This
pipeline moves real prune deletions into :commit_manifest -- i.e. only
after :verify has confirmed the newly-actuated project actually compiles
-- a strictly safer ordering (never delete a stale path until the new
state is confirmed good), disclosed here as a deliberate difference from
sync.ex's existing behavior, not an oversight.
Testing hooks (inert in real use)
Three optional, per-target opts keys exist ONLY to make :actuate's real
concurrency, and its compensation path, independently observable from a
test, and are otherwise inert no-ops:
:test_delay_ms--Process.sleep/1for this many ms immediately before this target's real write.:test_probe-- an atom naming a public ETS table; if given,{{index, :start}, monotonic_ms}and{{index, :stop}, monotonic_ms}are inserted around the (possibly delayed) write, so a test can assert two targets' real write windows actually overlapped.:test_chmod_after_write-- an integer POSIX mode (e.g.0o444); if given,File.chmod!/2is called with this mode against the target path IMMEDIATELY after this target's real write genuinely succeeds (deterministic, not timing-dependent::actuateis a separate, strictly-later Reactor step than:verify/:actuate's own undo, so the file is guaranteed read-only well before either ever runs). This is what lets a test construct a REAL "compensation itself fails" scenario without racing a background process against the pipeline: the real mutation genuinely happens first (this target's write succeeds normally), then the file becomes genuinely unwritable, then a later step's real failure triggers a real revert attempt against it -- see "Compensation failure::compensation_failed" below andtest/ggen_igniter_compensation_failure_test.exs.
None of the three is read anywhere outside actuate_one/2 below, and all
are nil (no-op) unless a caller deliberately sets them.
Compensation failure: :compensation_failed (the sixth -- catastrophic -- outcome)
Every rollback attempt this module makes (revert_all/1, called from both
:actuate's real undo/3 -- a LATER step failed -- and from
actuate_pending/2's own internal self-heal -- :actuate's OWN run
failed) used to call revert_one/2 directly and let a real revert failure
(a target that became read-only or was deleted out from under the
process) RAISE uncaught. Reactor's own do_undo/6
(deps/reactor/lib/reactor/executor/step_runner.ex) has NO rescue
around its call to Step.undo/4 -- confirmed by reading that file
directly, not assumed -- so a raise from inside undo/3 used to propagate
as a bare, uncaught exception all the way out of Reactor.run/4, past
this module's OWN run/1 case entirely. That is strictly WORSE than
mis-reporting :refused: no receipt was persisted at all, for an attempt
that genuinely mutated disk and then failed to restore it -- exactly the
silent-catastrophe class this module's evidence guarantee (correction A)
exists to close.
Fixed: revert_all/1 is now a real, best-effort, NEVER-RAISING operation
-- each path's own revert is individually rescued (revert_one_safe/2),
so one path's failure does not stop attempts on the others, and the
overall result is a tagged {:ok, restored_paths} or
{:error, %{paths:, restored:, failed:}} (failed :: [{path, {module, message}}]). Both call sites (:actuate's undo/3 and
actuate_pending/2's self-heal branch) now inspect this tagged result:
on :error, they emit a real COMPENSATION_FAILED OCEL event and return
{:error, {:compensation_failed, details}} from their OWN
run/undo-callback -- Reactor's documented, non-raising failure contract
({:error, reason}), never a raw exception. run/1's error branch
(find_compensation_failure/1) recursively searches the returned
Reactor.Error class for this specific tagged reason -- regardless of
which step name Reactor happens to attach it to (:actuate's undo
failing surfaces as an UndoStepError; :actuate's own self-heal failing
surfaces as a RunStepError for :actuate itself) -- and when found,
builds a GgenIgniter.Receipt with standing: :compensation_failed
whose reason names, explicitly and in one sentence: that a real
mutation occurred, that verification failed, that restoration
(compensation) itself ALSO failed, the exact paths that could not be
restored and why, which paths (if any) WERE restored, and that manual
repair may be required. See GgenIgniter.Receipt's moduledoc for this
standing's full contract, and test/ggen_igniter_compensation_failure_test.exs
for the real, no-mock proof (File.chmod!/2 on a real target makes a
real revert write fail, via the :test_chmod_after_write hook above).
Telemetry
middlewares do middleware Reactor.Middleware.Telemetry end wires
Reactor's own built-in middleware, emitting real :telemetry.execute/3
events for this reactor's run/step/compensate/undo timing under
[:reactor, :run, :start | :stop], [:reactor, :step, :run, :start | :stop],
[:reactor, :step, :guard, :start | :stop],
[:reactor, :step, :process, :start | :stop],
[:reactor, :step, :compensate, :start | :stop], and
[:reactor, :step, :undo, :start | :stop] -- see
test/ggen_igniter_reconcile_reactor_telemetry_test.exs for the real,
no-mock proof (a real :telemetry.attach_many/4 handler process
receiving real events from a real ReconcileReactor.run/1 execution).
Alongside Reactor.Middleware.Telemetry,
GgenIgniter.Reactors.CompensationTelemetryMiddleware is wired here too --
it turns three of the same real Reactor lifecycle events
({:compensate_start, _} / {:compensate_error, _} / :undo_start) plus
two real error/2-derived standings (:compensation_failed /
:build_broken, via the same find_compensation_failure/1/
find_step_error/2 helpers run/1 itself uses) into durable, real ETS
counters (CompensationTelemetryMiddleware.counters/0) instead of
ephemeral :telemetry events -- see that module's own moduledoc for the
full real-event-shape citations and
test/ggen_igniter_reconcile_reactor_compensation_telemetry_test.exs for
the real, no-mock proof.
Summary
Functions
Read-only admission preview backing mix ggen_igniter.plan
(Mix.Tasks.GgenIgniter.Plan -- see that module's moduledoc for the full
CLI contract). Runs the SAME observe-prior-manifest -> load-ontology ->
resolve-pack -> run-queries -> render -> admit sequence run/1 runs for a
real reconciliation -- reusing the exact same private helpers
(normalize_targets/1, run_target_queries/3, build_plan/3,
admit_pending/2) rather than a parallel plan-only implementation -- but
STOPS before :actuate/:verify/:finalize_evidence ever run: nothing is
written to disk, no GgenIgniter.Receipt is persisted, and no manifest is
promoted. This is a plain function, not a Reactor.run/4 invocation --
there is nothing here for Reactor's compensation/undo machinery to protect
against, since no mutation ever happens on this path.
The recommended entry point: runs one full reconcile attempt for
reconcile_opts (the same flat/:targets shape this module's steps
accept -- see moduledoc). Unlike calling
Reactor.run(__MODULE__, %{reconcile_opts: reconcile_opts}) directly,
this function GUARANTEES a real GgenIgniter.Receipt is persisted
(GgenIgniter.Receipt.append!/2) on every path: :alive on success
(already true of the bare :finalize_evidence step), and :refused /
:compensated / :build_broken on the three real failure paths, which
:finalize_evidence structurally cannot reach itself (it only runs after
:admit + :actuate + :verify all succeed). See moduledoc's
"Corrections applied" section (correction A).
Functions
@spec plan(keyword()) :: {:ok, [GgenIgniter.PendingActuation.t()]} | {:error, {:unsupported_capability, String.t()}} | {:error, term()}
Read-only admission preview backing mix ggen_igniter.plan
(Mix.Tasks.GgenIgniter.Plan -- see that module's moduledoc for the full
CLI contract). Runs the SAME observe-prior-manifest -> load-ontology ->
resolve-pack -> run-queries -> render -> admit sequence run/1 runs for a
real reconciliation -- reusing the exact same private helpers
(normalize_targets/1, run_target_queries/3, build_plan/3,
admit_pending/2) rather than a parallel plan-only implementation -- but
STOPS before :actuate/:verify/:finalize_evidence ever run: nothing is
written to disk, no GgenIgniter.Receipt is persisted, and no manifest is
promoted. This is a plain function, not a Reactor.run/4 invocation --
there is nothing here for Reactor's compensation/undo machinery to protect
against, since no mutation ever happens on this path.
Returns {:ok, [%PendingActuation{}]} -- the exact admitted plan :admit
would hand to :actuate, unwrapped from admit_pending/2's own
%{pending: pending, ...} map since callers of plan/1 (today, only
Mix.Tasks.GgenIgniter.Plan.report/3) only need the list itself.
{:error, {:unsupported_capability, reason}} when the resolved template
has a --- frontmatter header -- this bounded pipeline (like run/1,
via Mix.Tasks.GgenIgniter.Sync's own run_via_reactor/3 guard) does not
implement frontmatter parsing. {:error, reason} for any other
admission-time refusal (one of admit_pending/2's own tagged reasons:
:refused_duplicate_output_path / :refused_path_escapes_root /
:refused_unowned_delete / :refused_stale_outputs).
Raises ArgumentError for an unresolvable input (missing/ambiguous
template, ontology, or query) -- the same vocabulary resolve_ontology_path!/1
and resolve_template_path!/1 already raise for run/1; Mix.Tasks.GgenIgniter.Plan
rescues this itself and turns it into exit code 2, this function does not
catch its own raises.
@spec run(keyword()) :: {:ok, GgenIgniter.Receipt.t()} | {:error, GgenIgniter.Receipt.t()}
The recommended entry point: runs one full reconcile attempt for
reconcile_opts (the same flat/:targets shape this module's steps
accept -- see moduledoc). Unlike calling
Reactor.run(__MODULE__, %{reconcile_opts: reconcile_opts}) directly,
this function GUARANTEES a real GgenIgniter.Receipt is persisted
(GgenIgniter.Receipt.append!/2) on every path: :alive on success
(already true of the bare :finalize_evidence step), and :refused /
:compensated / :build_broken on the three real failure paths, which
:finalize_evidence structurally cannot reach itself (it only runs after
:admit + :actuate + :verify all succeed). See moduledoc's
"Corrections applied" section (correction A).