A streamed answer feels simple until two conversations run at once, the user changes screens, a token begins with a space, or the user presses Stop before the HTTP request has fully started. Add a bearer token and health-record context, and the transport boundary becomes part of the security and correctness design – not merely UI plumbing.
Part 6 follows one grounded answer from the Desktop App or Mobile App through a Tauri v2 Rust bridge, into a Rocket server, through local retrieval and Foundry Local, and back as correlated events. The pattern keeps the JWT, base URL, HTTP client, and SSE stream outside the web view while supporting concurrent runs, per-conversation queues, cancellation, bounded server memory, citations, and optional verification.
The transport boundary is also where a residency obligation is most easily undone. Where a country requires processing connected to primary or secondary health care to be carried out through a server and data centre inside its borders – Kenya, the setting of the research behind this series, is one – a phone in a ward does not breach that, provided the server it talks to stays inside the facility and the answer is assembled there. But a web view holding the JWT and calling out directly, a crash reporter that ships prompt text, or an analytics SDK bundled into the client would each move record content offsite. Keeping credentials, HTTP, and the SSE stream in the Rust bridge is what keeps the client a rendering surface rather than a second egress point. Part 1 sets out the boundary.
This is a prototype using synthetic health data. It is not compliance or clinical guidance, and the Mobile App is initialized and configured but not production-device validated.
This series implements the pattern in Rust, but it can use the Foundry Local SDK Reference with C#, JavaScript, Python, or Rust; Part 2 lists the language-specific links.
Problem statement
A chat UI becomes unsafe and inconsistent when its web layer holds a bearer token, calls the server directly, or treats all streaming work as one global loading state. Concurrent conversations, navigation, cancellation, and partial responses then compete for shared listeners and mutable state.
SSE adds protocol-specific risks: leading whitespace can be lost, listeners can outlive screens, and late events can update the wrong conversation. If the client also supplies authoritative history, a modified Desktop App or Mobile App can inject memory that the server never persisted or authorized.
The chat transport must put credentials and networking behind the Tauri Rust bridge, correlate every event to an isolated run, and let request lifetime survive navigation without leaking listeners. The Rocket server must own conversation memory and evidence, while cancellation and partial failures remain explicit to the client. No part of that path – request, stream, log, or crash report – may terminate outside the facility.
Success criteria
- The web view never stores the JWT or makes direct authenticated API calls.
- Every stream event is correlated to one run and one conversation.
- Concurrent runs, navigation, and cancellation cannot cross-update state.
- SSE token framing preserves whitespace and typed payload boundaries.
- The server authorizes and assembles bounded conversation memory.
- Partial output is distinguishable from a committed assistant message.
Put transport and credentials in the Rust bridge
The Desktop App and Mobile App share a Tauri shell and web-view implementation. The web layer renders the experience and manages presentation state. It does not call the server directly and never owns the JWT.
The local Rust bridge owns:
- the configured server base URL;
- the bearer JWT;
- the reqwest client;
- authenticated REST requests;
- SSE consumption;
- per-run abort handles; and
- translation from server events to typed Tauri events.
External PostgreSQL, MySQL, and SQL Server databases are not shown on the hot semantic-chat path. They are grouped as External Clinical Databases elsewhere in the architecture and feed the internal store through ingestion or serve separately guarded live SQL. The Internal DocumentDB Hybrid Store is not one of those sources.
The bridge boundary reduces accidental exposure: a component cannot read a token from browser storage or quietly invent a direct API call. It is not complete hardening, however. The current bridge persists the token in a plugin store rather than an OS secure credential store, and development transport defaults to HTTP. Production PHI deployment still requires TLS, approved certificates, least-privilege Tauri capabilities, CSP review, and secure device storage.
Stage 1: create a run, not a global “loading” flag
A single isStreaming boolean fails as soon as users can switch conversations. The client runtime instead generates a UUID for each submission and creates a run record containing its conversation, prompt, options, phase, partial answer, citations, structured data, and error state.
const runId = crypto.randomUUID();
addActive(activeChatRuns, conversationId, runId);
useChat.getState().startRun(runId, conversationId, prompt, options);
try {
await chat(prompt, [], options, conversationId, runId);
await queryClient.invalidateQueries({ queryKey: [‘messages’, conversationId] });
} finally {
removeActive(activeChatRuns, conversationId, runId);
startNextChat(conversationId);
}
Independent conversations can run concurrently. Within one conversation, normal prompts queue behind active work. A deliberate Send now action bypasses that queue, with an explicit consequence: the unfinished answer is not yet part of server-loaded history, so the new run may not see it.
Navigation does not own the request lifetime. An app-level conversation runtime does. Switching tabs or conversations therefore cannot orphan the stream. Rename and delete are blocked only where active work makes them unsafe; the rest of the application remains usable.
Stage 2: load server-owned working memory
When a conversation_id is present, the Rocket server verifies that the JWT subject owns it. It persists the user turn before answer generation, then assembles bounded working memory from DocumentDB:
- an optional rolling summary of older messages; and
- a chronological verbatim tail bounded by turn count and approximate word budget.
Long assistant turns are clipped before entering memory. The summary becomes a synthetic leading turn for query rewrite, while summary and tail can provide continuity to generation. They are not citable evidence.
After a successful assistant turn, write-behind compaction can fold older overflow into a concise local-model summary. Compare-and-swap on summary_upto prevents two overlapping compactions from interleaving. A database or summarization failure does not block the visible answer; later turns retain the bounded tail and can retry compaction.
This makes DocumentDB the authoritative transcript. PHI-bearing drafts and active buffers remain in client session memory rather than browser persistence.
Stage 3: choose the answer path before streaming tokens
The server’s tiered router chooses among conversational, conversation-meta, structured SQL, constrained DocumentDB aggregation, hybrid classification, and semantic retrieval. The semantic path from Part 5 performs rewrite and expansion, BGE-M3 embeddings, concurrent vector and lexical search, RRF, row dedupe, reranking, and a relevance gate.
If the gate rejects a pointed query, the server emits an explicit no-relevant-records answer instead of opening ungrounded generation. If evidence is accepted, a bounded prompt contains numbered retrieved passages. Only those passages may be cited.
Foundry Local then opens a local streaming chat completion. In this project the Rust server embeds the native SDK in process; there is no separate model HTTP endpoint and no cloud-model fallback. Missing local inference degrades explicitly rather than sending sensitive prompts elsewhere.
The server persists a successful assistant turn only after generation completes. A mid-stream failure can therefore leave the already-persisted user message without an assistant record. The client preserves partial visible output for retry, but it must not present that partial output as a committed transcript entry.
Stage 4: treat SSE as a framed protocol
The server emits typed events rather than one undifferentiated byte stream. Depending on the selected path, a chat run can carry:
- routed for the route decision;
- citations for semantic evidence;
- sql, columns, and rows for live-source results;
- token for visible answer fragments;
- verify for a post-stream report;
- error; and
- done.
A subtle but important detail is token encoding. A bare SSE line such as data: word can lose leading whitespace during parsing and fuse streamed words. The server JSON-encodes each token:
fn token_event(text: &str) -> Event {
let data = serde_json::to_string(text)
.unwrap_or_else(|_| “\”\””.to_string());
Event::data(data).event(“token”)
}
The bridge decodes that JSON string before emitting chat://token. Structured payloads remain JSON text until the typed frontend listener parses them.
The planned event order
The service-line design in plan 04 and plan 06 fixes one event order shared by /chat and every /agents/<kind> route, because both are meant to call the same structured executor:
routed
→ [clarify]
→ sql, columns, rows (live source SQL)
| spec, rows, pipeline (constrained DocumentDB aggregation)
| citations (semantic passages)
→ provenance
→ token*
→ [verify]
→ [suggestions]
→ done | error
Three things change relative to today’s stream. routed gains service_line, deterministic, and focus_used, so the client can label the answering department, mark answers that never touched a model planner, and show which remembered entities were substituted into the question. A clarify event may replace the evidence block entirely when the router is confident the question is structured but a required slot is unbound; it carries one question and a short option list, and the design allows at most one clarification per turn. And provenance becomes a first-class event that precedes the first token: an ordered list of rungs actually attempted – link, deterministic SQL, model SQL, validate, execute, aggregation, list, retrieval – each marked hit, miss with a PHI-free reason, or skipped, plus the backend used, the scope, and per-rung timings. Today the client infers provenance from which events happened to arrive; the planned contract states it.
The bridge relays the new events under the existing envelope as agent://provenance, agent://suggestions, and agent://clarify (with chat://* equivalents) and passes routed through unchanged. Each new field is also persisted on the assistant message, so a reloaded conversation shows the same provenance and suggestions the live stream did. None of this order is implemented yet; the current stream is the bullet list above.
The bridge stamps every event with the client-generated run ID:
#[derive(Serialize, Clone)]
struct ChatEvent {
run_id: String,
data: String,
}
app.emit(
“chat://token”,
ChatEvent {
run_id: run_id.clone(),
data: decode_token(&event.data),
},
)?;
Without the envelope, two simultaneous streams can interleave in one answer. Channel names identify event kinds; run IDs identify instances.
Stage 5: register listeners once at boot
Per-component listeners are tempting: mount a chat view, subscribe, and unsubscribe on navigation. That pattern leaks under hot reload, misses events when a route unmounts, and makes concurrent routing fragile.
The application instead installs one listener for each long-lived channel at boot. Each listener checks the run ID, then fans the event into a Zustand store. Token bursts are buffered per run and flushed on animation frames:
const tokenBuffer = createKeyedRafBuffer<string>((runId, batch) => {
useChat.getState().appendAnswer(runId, batch);
});
await listen<ChatEvent>(‘chat://token’, ({ payload }) => {
if (useChat.getState().runs[payload.run_id]) {
tokenBuffer.push(payload.run_id, payload.data);
}
});
await listen<ChatEvent>(‘chat://done’, ({ payload }) => {
tokenBuffer.flushNow(payload.run_id);
useChat.getState().finish(payload.run_id);
});
Per-run buffering prevents simultaneous answers from sharing a frame batch. Flushing before terminal state prevents the final token fragment from arriving after the UI marks the run complete.
The same global-listener pattern handles agent runs, ingestion progress, model downloads, and logs, using run_id, job_id, or variant_id as the appropriate correlation key.
Stage 6: cancellation must handle races
The bridge creates an abort handle before beginning the request and registers it under the run ID. cancel_run removes and aborts that handle. Logout aborts every active run before clearing session state.
There is also a narrow race: the user can press Stop before the invocation has registered its handle. The bridge keeps a bounded, time-limited cancelled_before_start map. When the late registration arrives, it is aborted immediately rather than resurrecting a stopped request.
pub fn register_run(&self, id: String, handle: AbortHandle) {
let mut runs = self.active_runs.lock().expect(“run registry poisoned”);
if runs.cancelled_before_start.remove(&id).is_some() {
handle.abort();
} else {
runs.active.insert(id, handle);
}
}
The map has a TTL and maximum size, preventing malicious or accidental unknown IDs from growing memory indefinitely.
Cancellation currently drops the bridge-side HTTP/SSE stream. That is useful and visible, but complete propagation through already-running server retrieval, SQL, blocking embedding, or native generation is not proven. Documentation should distinguish “the client stopped receiving” from “all server computation terminated immediately.”
Stage 7: persist evidence and verify the exact message
For semantic answers, citations are emitted before tokens and persisted with the assistant message. For live SQL, the source ID, validated SQL, columns, and exact rows are retained. For constrained DocumentDB aggregation, the specification, rows, and generated pipeline are retained. The evidence type follows the route instead of being flattened into prose.
Optional verification runs after streaming. Deterministic logic detects citation numbers outside the supplied range. A local verifier can then assess factual claims against the numbered passages. The report is persisted against the exact assistant-message ID, avoiding an unsafe “update the latest message” race when runs overlap.
Verification adds a signal, not a guarantee. It is off by default, can be skipped on timeout or malformed output, and does not retract text already displayed.
Planned: agents as service lines, and the UI that shows the evidence
The current agent tabs – auto, health_query, trends, patient_lookup, summarize, chat – are hardcoded in the web layer and named after techniques. The design in plans/new reframes them as hospital service lines: Ask, plus Patient Chart, Front Desk, Ward Board, Emergency, Maternity, Theatre, Pharmacy, Diagnostics, Revenue, Quality & Safety, Workforce, Chronic Care, and Facilities. Each line owns a set of hospital concepts; which physical tables it can read comes from the per-source schema binding, not from code. Every line offers the same three modes – Ask, Trends, and Handover – as a toggle rather than as separate tabs. Existing persisted conversations keep working through a server-side mapping of the legacy kinds onto Ask.
That changes what the client must render (plan 07 – app restructure):
- Registry-driven tabs. The tab bar is built from GET /agents, which returns each line’s label, tier, whether it is usable on the connected sources, its modes, its table scope per source, and example questions. Unusable lines are hidden or greyed; no service-line slug or label is a string literal in the web layer. The bridge’s agent kind becomes an opaque string validated server-side so a new line never needs a bridge rebuild.
- Scope panel. The side panel shows the agent’s blurb and “data this agent can see” as table chips grouped by source, drawn from the same registry payload.
- Provenance strip. A compact rung list under each assistant bubble – Link ✓ → Deterministic SQL ✓ → Validate ✓ → Execute ✓ · 212 ms – with misses showing their PHI-free reason on hover, and an expansion revealing the SQL and explanation, the aggregation spec and pipeline, or the retrieval filter as appropriate.
- Suggestion chips. Up to four follow-ups under the last assistant message only. They are generated by code from a mutated query specification that already binds, so clicking one should never produce a clarify or a miss. A click is a normal turn: the text is persisted as the user message, and the pre-parsed spec travels alongside so the router can skip parsing.
- Clarify blocks. The clarify question renders with its options as buttons that submit the option as the next prompt; a free-text reply works too.
- Focus and mode badges. A small “using: patient · period” chip when focus_used is non-empty, and the mode badge on each answer.
Draft and conversation continuity are part of the contract, not polish. Switching tabs never discards the composer draft; drafts are keyed by conversation, or by kind for a not-yet-created one. Re-entering a tab restores its last open conversation. A conversation can move between agents (“Open in Maternity”) and its focus and history travel with it, with each bubble badged by the agent that answered. Today setSelectedKind resets the active conversation; the planned store keeps it.
All of this is design direction. The mirrored-contract rule from the tradeoffs below applies with more force, because the number of fields the bridge and TypeScript must carry roughly doubles.
Tradeoffs and anti-patterns
The bridge adds code and mirrored contracts. Every new response field or event must be represented in Rust and TypeScript or it can disappear silently. That duplication is preferable to letting the web view own credentials and arbitrary networking, but it requires contract tests and disciplined updates.
Avoid these anti-patterns:
- storing the JWT in ordinary web storage;
- calling the server directly from React components;
- registering stream listeners per component or per request;
- using one global partial-answer string;
- emitting unencoded SSE token text;
- treating route navigation as cancellation;
- persisting an assistant turn before its stream succeeds;
- updating “the latest” message when concurrent runs exist;
- claiming cancellation interrupts every native stage without evidence.
Prototype status
Implemented: Desktop App flow; shared Mobile App codebase and Android project; bridge-owned base URL, JWT, HTTP, and SSE; JSON token events; boot-time listeners; per-run registries; per-conversation queues; Send now; retry; bridge cancellation; logout-wide abort; server-owned summary-plus-tail memory; grounded citations; local Foundry Local generation; and optional post-stream verification.
Partial: Mobile App device validation, secure OS-backed token storage, production TLS/certificate posture, CSP/capability hardening, and end-to-end cancellation through all server/native work. Not started: the service-line agent registry, provenance/suggestions/clarify events, and the UI elements described above. Tests use synthetic health records and do not establish clinical correctness or regulatory compliance.
Key takeaways
- Make the Rust bridge the credential and transport boundary.
- Correlate every stream event with a run ID.
- Own request lifetimes above route components and queue per conversation.
- JSON-encode SSE tokens to preserve whitespace.
- Register global listeners once and batch tokens independently per run.
- Persist only successful assistant turns with their exact evidence.
- Describe cancellation and Mobile App readiness no more strongly than validated behavior.
Continue the series
- Previous: Part 5 – A local hybrid retrieval pattern
- Next: Part 7 – Deterministic SQL before local LLM planning
Learn more on Microsoft Learn
- What is Foundry Local?
- Get started with Foundry Local
- Foundry Local SDK Reference – Rust on Windows
- Design and develop a RAG solution
The code behind this series is open source: github.com/kevin-gatimu/onprem-health-rag. It is a synthetic-data research prototype, not a clinical or compliance-ready system – issues and corrections are welcome.