GgenIgniter.EngineComparisonReport (ggen_igniter v26.9.8)

Copy Markdown View Source

The runtime, per-invocation, N-way surfacing mechanism ADR-0008 introduces -- NOT a new correctness oracle. pairwise_agreement/1 restates, over real rows returned from a real run, exactly the row-set/row-order/error-shape distinction classes test/ggen_igniter_cross_engine_equivalence_properties_test.exs and test/ggen_igniter_engine_parity_test.exs already establish statically:

  • Row-SET equality via MapSet.new/1 over each candidate's raw rows (mirroring ggen_igniter_cross_engine_equivalence_properties_test.exs's own rows_as_set/1 method, for the identical reason that test names -- order is a separately-tracked, already-known-divergent axis).
  • Row-ORDER equality as a SEPARATE raw-list == check, deliberately kept apart from set equality per the confirmed sparql-hex ORDER BY row-reversal bug documented in lib/ggen_igniter/query.ex:4-16's moduledoc. A future divergence of that exact shape shows up here as order_equal?: false with row_set_equal?: true, never collapsed into one ambiguous boolean.

Two structs:

  • t() -- one comparison run: query (the SPARQL text run, verbatim), candidates ([CandidateResult.t()], one per engine GgenIgniter. EngineRegistry.run_all/4 fanned out to), pairwise_agreement (a map keyed by {engine_a, engine_b} engine-atom pairs, engine_a ordered before engine_b per GgenIgniter.Engine.valid_names/0's own sort so a pair never appears twice under swapped keys), generated_at (DateTime.t()).
  • CandidateResult.t() -- one engine's real outcome: engine (atom), status (:ok | :error | :timeout), rows ([map()] | nil, nil only when status != :ok), row_count, elapsed_us (System.monotonic_time(:microsecond) delta; always a real non-negative integer, even on :error/:timeout, so the report can show how long a failing engine took before it failed), error (String.t() | nil, the real Exception.message/1 or exit-reason text on a non-:ok status).

to_markdown/1 and to_json/1 render either one report or a list of reports (GgenIgniter.EngineRegistry.run_all/4's caller runs one report per named --query; mix ggen_igniter.sync --engine oxigraph,sparql with more than one --query name=path.rq produces more than one report over one invocation). No new templating path: to_markdown/1 follows this repo's existing GgenIgniter.Render.render/2 convention (lib/ggen_igniter/ render.ex:8-11, stdlib EEx.eval_string/2 against bindings) rather than inventing a second one. to_json/1 uses Jason (already a dependency, see mix.exs's {:jason, "~> 1.4"}) -- no new dependency either.

Summary

Types

t()

Functions

pairwise_agreement(candidates)

Computes t().pairwise_agreement over every status: :ok pair in candidates (a status: :error/:timeout candidate has no real rows to compare, so it is excluded from every pair rather than silently compared against nil). Every unordered pair is computed exactly once, keyed {engine_a, engine_b} with engine_a preceding engine_b in the input list's own order (never both {a, b} and {b, a}).

to_json(report)

Renders one report, or a list of reports (see moduledoc), as pretty-printed JSON.

to_json_map(report)

The plain, JSON-safe map to_json/1 encodes -- exposed separately so a caller (e.g. mix ggen_igniter.sync's --engine-report) can wrap it (adding a "query_name" field) without re-parsing the encoded string.

to_markdown(report)

Renders one report, or a list of reports (see moduledoc), as a Markdown document (stdlib EEx.eval_string/2, GgenIgniter.Render.render/2's own convention -- no second templating path).

Types

t()

@type t() :: %GgenIgniter.EngineComparisonReport{
  candidates: [GgenIgniter.EngineComparisonReport.CandidateResult.t()],
  generated_at: DateTime.t(),
  pairwise_agreement: %{
    required({atom(), atom()}) => %{
      row_set_equal?: boolean(),
      row_count_diff: integer(),
      order_equal?: boolean()
    }
  },
  query: String.t()
}

Functions

pairwise_agreement(candidates)

@spec pairwise_agreement([GgenIgniter.EngineComparisonReport.CandidateResult.t()]) ::
  %{
    required({atom(), atom()}) => %{
      row_set_equal?: boolean(),
      row_count_diff: integer(),
      order_equal?: boolean()
    }
  }

Computes t().pairwise_agreement over every status: :ok pair in candidates (a status: :error/:timeout candidate has no real rows to compare, so it is excluded from every pair rather than silently compared against nil). Every unordered pair is computed exactly once, keyed {engine_a, engine_b} with engine_a preceding engine_b in the input list's own order (never both {a, b} and {b, a}).

Examples

iex> a = %GgenIgniter.EngineComparisonReport.CandidateResult{engine: :oxigraph, status: :ok, rows: [%{"s" => "x"}], row_count: 1}
iex> b = %GgenIgniter.EngineComparisonReport.CandidateResult{engine: :sparql, status: :ok, rows: [%{"s" => "x"}], row_count: 1}
iex> GgenIgniter.EngineComparisonReport.pairwise_agreement([a, b])
%{{:oxigraph, :sparql} => %{row_set_equal?: true, row_count_diff: 0, order_equal?: true}}

to_json(report)

@spec to_json(t() | [t()]) :: String.t()

Renders one report, or a list of reports (see moduledoc), as pretty-printed JSON.

to_json_map(report)

@spec to_json_map(t()) :: map()

The plain, JSON-safe map to_json/1 encodes -- exposed separately so a caller (e.g. mix ggen_igniter.sync's --engine-report) can wrap it (adding a "query_name" field) without re-parsing the encoded string.

to_markdown(report)

@spec to_markdown(t() | [t()]) :: String.t()

Renders one report, or a list of reports (see moduledoc), as a Markdown document (stdlib EEx.eval_string/2, GgenIgniter.Render.render/2's own convention -- no second templating path).