“On premises” is a placement decision, not a security certification. Keeping prompts and records off cloud model endpoints removes an important data-transfer path, but it does not protect credentials on disk, traffic on the LAN, overprivileged users, overloaded accelerators, verbose errors, or incomplete audit trails.
A useful hardening review therefore starts with trust boundaries and distinguishes three categories:
- controls implemented in code;
- controls that are present but limited; and
- controls that must block production use until completed and verified.
This synthetic prototype uses a Desktop App and Mobile App, a Tauri Rust bridge, a Rocket server, external clinical databases, an internal DocumentDB hybrid store, and local models. No health-record prompt or output is intentionally sent to a cloud model.
The series implementation uses Rust, though the pattern can be implemented with the Foundry Local SDK Reference in C#, JavaScript, Python, or Rust; Part 2 contains the language links.
It is worth stating precisely what locality does settle, because the rest of this article is about what it does not. Because the corpus, the index, the embedding computation, and language-model inference all execute inside the facility, the architecture removes any reliance on cross-border transfer safeguards and satisfies an in-country processing requirement – the kind some countries impose on primary and secondary health care, Kenya among them – by construction: no copy of the record set exists outside the facility. That is an architectural property rather than a procedural undertaking, and it is the design’s principal privacy contribution.
Everything else has to be earned. Locality relocates the trust boundary; it does not secure what is inside it. An impact assessment is still required wherever the processing counts as high-risk, and regulators classify AI-powered clinical systems that way – which is a reason to treat the assessment as a design gate rather than a post-implementation formality. An authenticated insider reading beyond clinical need, credential compromise, an unencrypted LAN, and an audit log that is best-effort rather than tamper-evident are all untouched by where the server sits.
Prototype scope: this is a synthetic engineering prototype, not a compliance claim. Production approval requires an organization-specific threat model, legal and privacy review, operational controls, and measured evidence.
Problem statement
Running inference locally removes one cloud transfer path, but it does not make a health-record application secure or available. PHI can still cross an unencrypted LAN, appear in verbose logs, persist beside weakly protected tokens, or become exposed through broad client capabilities and overprivileged database accounts.
Operational weaknesses compound those risks. Unbounded model or ingestion work can deny service, liveness can remain green while dependencies are unusable, and incomplete audit or backup practices can prevent investigation and recovery. Calling the deployment “on premises” does not resolve identity, transport, secrets, authorization, or capacity controls.
Hardening must layer code controls with deployment and operational controls, distinguish implemented safeguards from limitations, and name blockers that prohibit production claims. The result must fail closed at trust boundaries while remaining observable without logging sensitive content. It must also leave the residency property intact: a log shipper, an APM agent, or a crash reporter with an offsite destination re-creates the cross-border transfer the architecture was chosen to avoid.
Success criteria
- Identity and authorization are enforced server-side for every protected operation.
- Production transport authenticates endpoints and encrypts all sensitive links.
- Tokens and source credentials use approved storage and rotation mechanisms.
- Admission and readiness expose bounded capacity and dependency health.
- Logs, metrics, and errors omit prompts, records, secrets, and identifying content.
- Unresolved PHI, recovery, or authorization gaps are documented as release blockers.
Begin with explicit trust boundaries
The web layer does not call the server directly and does not hold the JWT. It invokes Tauri commands. The Rust bridge owns the base URL and token, adds authorization headers, and relays server-sent events. This reduces accidental credential exposure to frontend state and browser APIs, but it does not make the bridge’s storage or transport safe by itself.
The diagram intentionally separates External Clinical Databases from the Internal DocumentDB Hybrid Store. The former remain systems of operational record. The latter is application-owned storage for copied text, vectors, metadata, generations, conversations, users, audit entries, and other state. Treating them as one undifferentiated “database” obscures different credentials, retention rules, and authorization obligations.
Identity controls that exist today
The current server has a meaningful authentication baseline:
- Passwords use Argon2id with a random salt per password.
- Unknown users still trigger a dummy Argon2 verification, reducing username-enumeration timing differences.
- Login accepts username or email and returns the same unauthorized response for an unknown user and a wrong password.
- A process-local, per-IP throttle starts locking after five failed attempts, doubles the lock duration after further failures, caps it at 15 minutes, and resets after idle time.
- JWTs use HS256 and include user identity, role, issued/expiry times, and a token-version counter.
- Every authenticated request verifies the JWT, reloads the user, and compares the stored token_version with the token claim.
- Logout, force logout, password changes, password resets, and actual role changes increment token_version, invalidating outstanding tokens.
- Four roles exist: admin, doctor, nurse, and analyst.
- Administrative routes enforce admin access server-side.
- Conversation access is scoped to the authenticated owner and returns 404 on an ownership mismatch.
- Last-admin and self-delete guards protect account administration.
This design follows useful Zero Trust themes: verify identity on every request, minimize privilege, and assume a signed token may need revocation before expiry. Client-side route visibility is only presentation; Rocket guards remain the security boundary.
There are still limitations. Throttling is process-local and IP-based, so it is not suitable for a clustered deployment. JWT validation does not currently enforce issuer, audience, or token ID. Password-strength rules and proactive UI expiry timers are not implemented.
More importantly, role checks and conversation ownership do not yet constitute general record-level authorization. A production system needs an explicit policy for which patients, facilities, departments, source tables, and fields each caller may access. That open item cannot be solved by prompt instructions.
Secrets: encrypted does not mean fully managed
Source-database passwords are encrypted before storage with AES-256-GCM and a fresh random 96-bit nonce. The stored representation combines nonce and authenticated ciphertext. Key material is domain-separated with SHA-256 from ONPREM_CREDENTIALS_KEY; development can fall back to the JWT secret with a warning.
Production startup fails when critical secrets are placeholders, too short, absent, or improperly shared. This is a valuable fail-fast control. Environment files and the bridge store are excluded from Git.
The remaining gaps are operationally significant:
- key derivation has not migrated to HKDF;
- ciphertext does not carry a key version with a versioned decrypt path;
- there is no re-encryption command for credential-key rotation;
- the JWT and base URL are persisted in plaintext bridge-store.json rather than Windows Credential Manager or Android Keystore-backed storage; and
- the webview capability set still grants broad store access that needs least-privilege review.
A secure migration needs a versioned envelope, old/new key overlap, an auditable rewrap process, and tested rollback.
Transport is the largest current blocker
The development default is plain HTTP from the bridge to Rocket. That exposes credentials, bearer tokens, and PHI to a network observer. Production must terminate TLS with a trusted certificate and configure clients with an approved HTTPS hostname.
The DocumentDB development URI enables TLS while accepting invalid certificates. Encryption without server authentication permits interception; production must trust the correct CA and reject invalid certificates.
The Mobile App has another explicit development compromise: Android cleartext traffic is permitted globally so a runtime LAN IP can be entered. The commented examples for HTTPS or pinning are not enforcement. A production Android network policy must deny cleartext by default and permit only approved HTTPS destinations. Certificate pinning can reduce interception risk but creates certificate-rotation and recovery obligations; use it only with a supportable lifecycle.
The webview currently has no Content Security Policy. A strict CSP, constrained navigation and URL handling, and reduced Tauri capabilities are required before real PHI is introduced.
Admission control is a security and reliability boundary
Model inference, embedding, retrieval, and ingestion consume finite CPU, memory, GPU/NPU capacity, database connections, and file bandwidth. Without admission, a burst can become a self-inflicted denial of service.
Which accelerator is in use changes the shape of that limit rather than only its size. A host whose only registered provider is CPU has far less headroom than one with a discrete GPU, and one specific provider is worse than slow: ONNX Runtime’s WebGPU backend corrupts its own device state under concurrent generation and fails outright on prompts past roughly 20 KB, which is why generation concurrency is pinned to one and prompt size is capped (plans/docs/foundry-local-webgpu-concurrency-crash.md). Admission limits are therefore a property of a named host and its registered providers, not a constant to copy between sites.
The server uses timed semaphores for:
- active generations;
- active retrieval requests;
- active ingestion jobs globally; and
- active ingestion jobs per source.
When capacity is unavailable within the configured wait, the server returns 429 with a retry hint. This bounds the waiting population and gives clients an explicit backpressure signal.
Current admission is global rather than per-user or per-role. It prevents unlimited work but does not guarantee fairness. A production policy may need per-principal quotas, weighted classes, ingestion maintenance windows, or reserved capacity for clinical priority paths. Those choices should be measured rather than guessed.
Liveness and readiness answer different questions
GET /health is liveness-oriented: the process can respond, and it reports DocumentDB degradation without necessarily declaring the process dead.
GET /ready is stricter. It checks:
- DocumentDB access;
- required search indexes;
- Foundry availability;
- background warmup state; and
- free generation, retrieval, and ingestion capacity.
Readiness returns a non-ready status when required dependencies, initialization, or capacity are unavailable. That enables an operator or supervisor to stop sending work to a process that is alive but incapable of serving the intended request path.
A subtle tradeoff is capacity semantics. If readiness becomes false whenever permits are momentarily occupied, a load balancer can create oscillation. Validate the behavior with the actual supervisor and queueing policy.
PHI-safe observability requires data minimization
The request instrumentation is deliberately metadata-oriented. A RequestTrace records fields such as:
- endpoint and generated request ID;
- route, router tier, and cache hit;
- candidate counts before and after reranking;
- top rerank score and refusal-gate result;
- token-event and output-character counts;
- time to first token and total latency;
- per-stage durations;
- outcome and error class.
It does not intentionally retain prompt text, record text, JWTs, source credentials, or generated output text. The implementation counts output characters rather than storing output. This is the correct default for metrics in a PHI-sensitive system: collect what is needed to operate the service, not what is easiest to print.
An administrator-only /metrics/summary exposes process-lifetime request and outcome counts, bounded-window p50/p95/p99 latency and TTFT, and average stage durations. These metrics reset on restart and are not a durable monitoring backend.
Live logs are separate. An in-process hub retains 300 recent lines, broadcasts through a bounded channel, and feeds an authenticated /logs/stream; the client keeps a bounded 1,000-line view. These logs are ephemeral operational visibility, not an audit record. They have no file rotation, export, restart persistence, or retention policy. Although request summaries are designed not to include PHI, all log-producing call sites and third-party messages still need redaction review.
A useful logging rule is:
record identifiers, durations, counts, route labels, and error classes;
do not record prompts, row bodies, citations, tokens, passwords, or bearer tokens.
Hashing a patient identifier is not automatically anonymization. Prefer opaque request IDs and aggregate measures unless a specific operational need justifies more.
Data minimisation in the planned service-line design
The next design iteration (plans/new) introduces a per-source schema binding, service-line agents, and a streamed provenance event. Each creates a new place where PHI could leak if the rules are not written down before the code is. These are the intended rules; none are implemented yet.
- No PII in the binding. The binding records, for low-cardinality columns, a bounded list of distinct values so the router and SQL compiler can recognise “cancelled” or “ICU”. That list is populated only for enumeration-style roles (status, category, type, severity, priority, gender) and never for columns the binder or the ingest PII analyser has marked as contact details, identifiers, or person names. A binding should be safe to show any administrator and to include in a bug report. See plan 01.
- Listing queries do not select contact or identifier roles by default. When a deterministic listing is compiled from the binding, its projection is built from column roles, and the Contact and Identifier roles are excluded unless an administrator names the column explicitly. A question such as “list the patients on ward 3” should return names and business identifiers, not phone numbers and national IDs, because nothing in the question asked for them.
- Provenance payloads are PHI-free. The provenance event and its persisted copy describe the path taken – rung names, hit/miss/skipped, reasons like “bind error: no event time on Bill”, timings, the scope’s table names, the source ID. It never carries row values, question text, or generated text. The same restriction applies to the digest of the last result kept in the conversation focus, which records shape, counts, and a few short non-person labels only.
- Ownership scope is a read allow-list, not authorization. A service line’s table scope narrows what its linker, aggregation planner, and retrieval filter will look at. It does not decide what the caller may see. RBAC in auth/ is unchanged by the design, and the record-level authorization gap described above remains open. A scope that hides billing tables from the Maternity agent is a relevance and cost control; a user who should not see billing data needs a policy that the server enforces regardless of which tab they click.
The last point deserves emphasis because the UI will make scope visible – a “data this agent can see” panel is easy to misread as a permission statement. Documentation and the panel’s own wording should say what it is.
Audit is not the same as logging
The application writes security-relevant events to an audit_log collection with user ID, username, action, resource, optional details, and timestamp. Current call sites cover successful and failed login, logout and force logout, user administration, profile/password changes, source mutations, ingestion start, and wired NL-to-SQL execution events. An admin-only paged endpoint supports action, username, and time filters.
This is implemented, but durability and scope are incomplete:
- writes are best-effort and do not fail the primary operation;
- audit writes are not transactional with the mutation;
- there is no tamper-evident chain or append-only database permission policy;
- export, archival, retention, and incident-review procedures are absent;
- live trace logs are process-memory only and cannot substitute for audit; and
- the trail is not a comprehensive per-record authorization/access ledger.
For production, decide which events are mandatory and whether failure to record them should fail the primary action. Add integrity controls, restricted write/read identities, retention and legal-hold rules, export, time synchronization, and restoration tests.
Error handling must not leak internals
The server correctly logs 5xx errors on the server side, but the response serializer currently returns self.to_string() for every error. Database and internal variants can therefore expose implementation details to clients. Sanitized 5xx bodies remain open work.
The desired split is straightforward:
{
“error”: “internal server error”,
“request_id”: “opaque-correlation-id”
}
Operators use the request ID to find the detailed local log. Callers receive stable, non-sensitive text. Expected 4xx errors can remain specific where that does not reveal protected data; ownership failures should continue to avoid confirming another user’s resource exists.
User-derived regular expressions also require attention. Prefix filters and the administrative audit username filter currently permit unescaped regex syntax in some paths. Escape literal inputs, cap lengths, and avoid exposing general regex evaluation unless it is an intentional, authorized feature.
Production hardening checklist
Identity and authorization
- ☐ Preserve Argon2id, dummy verification, generic login failures, JWT verification, and token-version revocation.
- ☐ Define record-level authorization across patient, facility, source, table, and field boundaries.
- ☐ Use least-privileged read-only accounts for external clinical databases.
- ☐ Decide whether issuer, audience, token ID, stronger password policy, and per-account throttling are required.
- ☐ Test role changes, password changes, force logout, deleted users, and ownership failures.
Secrets and clients
- ☐ Move tokens to Windows Credential Manager and Android Keystore-backed storage.
- ☐ Remove unnecessary webview store and system capabilities.
- ☐ Add a strict CSP and safe markdown/link handling.
- ☐ Introduce HKDF plus versioned ciphertext and test key rotation.
Networks and dependencies
- ☐ Terminate TLS in front of Rocket with a trusted certificate.
- ☐ Deny Android cleartext and allow only approved HTTPS hosts.
- ☐ Remove invalid-certificate acceptance from DocumentDB and trust its CA.
- ☐ Test certificate renewal, expiry, hostname mismatch, and dependency outage behavior.
Operations and evidence
- ☐ Tune admission limits on named hardware and record rejection behavior.
- ☐ Validate /ready semantics with the actual supervisor or load balancer.
- ☐ Sanitize 5xx responses and escape regex-derived filters.
- ☐ Review every log source for PHI and secret leakage.
- ☐ When the schema binding and provenance event ship: assert no PII-role values in bindings, no contact/identifier roles in default listing projections, and no row or prompt text in provenance payloads.
- ☐ Make required audit events durable, integrity-protected, retained, and reviewable.
- ☐ Prove backup restoration and incident procedures with synthetic data.
Anti-patterns to avoid
- “It is on our LAN, so HTTP is acceptable.” Internal networks can be observed or compromised.
- “The JWT is outside React, so token storage is solved.” Plaintext bridge storage remains recoverable from the endpoint.
- “RBAC means records are authorized.” Coarse roles do not define patient- or facility-level access.
- “The agent’s scope is its permission.” A service-line table allow-list limits what an agent reads for relevance; it is not an access-control decision.
- “Encrypted database traffic is authenticated.” Accepting any certificate defeats server identity.
- “Logs are internal, so prompts are safe to record.” Internal logs are frequently copied, searched, exported, and retained.
- “Best-effort audit is a compliance ledger.” It can lose events and lacks tamper evidence.
- “Readiness proves production readiness.” It proves a current serving condition, not capacity, recovery, security, or clinical quality.
Tradeoffs
Security controls can reduce convenience. OS secure storage complicates cross-platform development. Certificate pinning increases rotation risk. Durable audit can add write latency and failure modes. Per-user quotas may underutilize hardware. Sanitized errors slow ad hoc debugging.
Those costs should be handled with explicit operator tooling, correlation IDs, documented break-glass procedures, and staged rollout – not by exposing PHI or weakening identity. The objective is not maximum restriction; it is a defensible allocation of trust.
Prototype status
Implemented: Argon2id, generic authentication failures with dummy verification, process-local login throttling, HS256 JWTs, token-version revocation, RBAC, conversation ownership, AES-256-GCM source-credential encryption, production secret checks, bounded admission, readiness, metadata-only request summaries, bounded process metrics, ephemeral live logs, and a best-effort audit trail.
Open or incomplete: trusted TLS for clients, trusted DocumentDB certificates, OS secure token storage, CSP and capability reduction, sanitized 5xx responses, HKDF/versioned encryption migration, production Android network policy, durable/tamper-evident audit, and general record-level authorization. No production security claim should be made until these are addressed and verified in the target environment.
Key takeaways
- On-premises placement reduces data movement; it does not replace security engineering.
- Keep credentials out of the web layer, then secure their persistence and transport too.
- Verify identity on each request and define authorization below the coarse role level.
- Treat admission and readiness as part of the protection boundary for scarce local inference.
- Make observability useful through counts, timings, and opaque IDs – not prompt or record text.
- Separate ephemeral operational logs from durable, integrity-protected audit evidence.
- Publish open risks as release blockers, not footnotes.
Source deep dives
- Security, authentication, and audit
- Operations, performance, and observability
- Architecture and data model
- Application and feature architecture
- Evaluation, production validation, and release
- Code: onprem-rag-server/src/auth/, onprem-rag-server/src/admission.rs, onprem-rag-server/src/routes/health.rs, onprem-rag-server/src/telemetry.rs, onprem-rag-server/src/logstream.rs, onprem-rag-server/src/crypto.rs, and onprem-rag-server/src/error.rs
Learn more on Microsoft Learn
- Secure app identity with permissions and access
- What is Foundry Local?
- Foundry Local SDK Reference – Rust on Windows
- Design and develop a RAG solution
Continue the series
Previous: Part 8 – Bounded Local Agentic Patterns
Next: Part 10 – Evaluate Before Production
The code behind this series is open source: github.com/kevin-gatimu/onprem-health-rag. It is a synthetic-data research prototype, not a clinical or compliance-ready system – issues and corrections are welcome.