Optional local Llama workbench

The online Edge Evidence Visual Gateway lets visitors inspect three synthetic example records without a local model or a running Mac. Inspection is deterministic. The Gateway combines Inspect, retained Proof and an Ask view explaining the optional local model. The hosted Ask view does not run inference. The separate Processor and Replication proof is a retained synthetic JSON result; it is not run when a visitor inspects a record.

The optional local workbench adds Llama 3.1 8B explanations on the reviewer’s own computer. It does not expose a hosted Llama backend, make fresh evidence authoritative, execute Processor or Replication, verify stored bytes, approve mutations, or produce a continuously deployed Capture → workers → model pipeline. A recording is optional and is not an interactive hosted model.

Requirements and setup

Use the public workbench source, pinned at 692713d2083033719bd68c1a17b67c0c0d6b43d3, with its checked-in requirements lock and source manifest. The exact Ollama pin at that revision is 0.40.11. The historical Mac baseline used Python 3.12.13, Ollama 0.40.6 and an already-installed llama3.1:8b with exact digest 46e0c10c039e019119339687c3c1757cc81b9da49709a3b3924863ba87ca666e; the 26 October check below used the same model digest with Ollama 0.40.11. The lock’s added dependency hashes are for Python 3.12 on macOS arm64. A different environment or candidate source needs its own verification. The app does not pull models, choose cloud fallback models or require cloud/API credentials.

From the workbench directory (examples/local-llama-workbench/ at that revision), with its runtime/, dependencies/, public-source/ and SOURCE-MANIFEST.json entries present:

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install --require-hashes --requirement dependencies/requirements.lock
export EDGE_EVIDENCE_OLLAMA_BIN="$(command -v ollama)"
export EDGE_EVIDENCE_MODEL_STORE="$HOME/.ollama/models"
python -B runtime/serve_panel.py

Open only the http://127.0.0.1:<printed-free-port>/ address reported by the server. This is a loopback-only workbench, not a public service or a feature to port-forward. The reviewed Mac arrangement leaves an existing Ollama listener on port 11434 alone and exclusively owns a separate 127.0.0.1:11435 model server; if that experimental port is busy, startup refuses to attach to or terminate the existing process. Warm model residency is finite (reviewed profile: up to ten idle minutes). The app verifies the pinned source and model rather than accepting a similar-looking replacement.

Inspect, Proof and Ask mean different things

1. Inspect: select one of three independent synthetic records. The server provides the record and separately computed deterministic inspector facts, including classification and refusal of mutation authority. Check the record’s raw fields, digest and response when explaining a result.
2. Proof: view explicitly retained Processor/Replication proof bytes and provenance. The original worker execution time is unknown and the source result may be cached; page viewing performs no new worker run or independent readback.
3. Ask: optionally submit a fresh, independent local-model turn about the frozen selected record. The model may make bounded, actual calls to a read-only inspector tool. Its provisional stream is not the authoritative record; only the constrained final display result is presented as an optional explanation. Different questions are not joined into a remembered conversation.

The runtime binds the selected source/fixture, tool and endpoint on the server, not in a browser toggle. The baseline caps user questions at 2,000 characters, model work to two requests and two read-only tool calls, context to 4,096 and output to 256 tokens, with an approximately 30-second turn deadline. It refuses unsafe hosts/origins, cross-site requests, unexpected fields/tools and source/model mismatches. These application guards and the independent inspector’s read-only contract are two distinct authority layers, not an operating-system sandbox or a guarantee that generated text is true.

Failure, cancellation and cleanup

An answer may be awkward, wrong, delayed, withheld or unavailable. The visible UI must distinguish provisional text, finalized guarded text, fallback deterministic inspection, model failures and explicitly mocked transport. Mock-only testing is not model acceptance. Press Cancel to stop an active turn; another turn remains blocked until the app verifies owned worker/model cleanup. Cancellation stops the owned experimental model server too; readiness becomes cold. Use Prepare model to rewarm before the next question. Closing a tab does not immediately cancel inference. Never kill or reuse an unrelated Ollama process.

Verification and evidence limits

The exported workbench passed 109 offline Python checks (four optional checks skipped) and five JavaScript tests. An earlier bounded real-Ollama browser check, recorded with the workbench’s 20 October 2026 source review, produced two accepted explanations and one withheld explanation across three selected questions, with approximately 9–11 seconds elapsed per turn.

At revision 692713d, the offline checks recorded 63 runtime checks passed (two optional historical-replay checks skipped), 48 workbench tests passed and five JavaScript tests passed, all with mocked transport. A second bounded real-inference check ran on 26 October 2026 at workbench head 963d9c6 with Ollama 0.40.11. Three questions were fixed in advance, each asked once, with no retries. It displayed one model explanation and two withheld answers, at roughly 4.6–11.7 seconds a turn. In both withheld turns, the retained raw model text expressed the relevant boundary in phrasings outside the guard’s recognised denials. The displayed explanation was accurate but not a complete answer.

A third bounded check the same day, at the merged revision 692713d, asked for plain explanations. A request to summarise the complete record displayed a complete, accurate two-sentence explanation (5.1 seconds). A request to explain the prepared record to a newcomer was withheld; its raw text wasn’t retained. A third question was asked against the wrong record by mistake and isn’t counted.

These observations are not a benchmark, a complete eleven-case rerun, a general safety finding, or proof that the guard detects every incorrect answer. Streaming may arrive late or in bursts. Preserve the deterministic baseline chapter’s existing public source pin; do not silently rebase historical research claims onto the workbench release.

For the baseline three-service and hosted arrangement see Run locally, Deterministic inspector baseline and Boundaries and verification. Original workbench source and synthetic assets are Apache-2.0. Model weights and dependencies retain their own terms; no weights are redistributed.