Security at Infer
Summary
Infer's security model starts from a structural decision: customer content is never stored. Prompts and completions exist only in volatile memory for the life of a request and its session cache, are excluded from logs by construction, and are not reachable by any internal tool or person. Most of a security program defends data at rest; Infer's first line of defense is that there is no data at rest to defend.
Architecture & data flow
An inference request follows one path:
- Routing partner → gateway. Requests arrive at Infer's gateway over TLS from our routing partner, the sole authorized caller, authenticated with an issued credential and, optionally, IP allowlisting. End users never connect to Infer directly.
- Gateway → compute. The gateway applies admission control and routes the request to a model replica over an encrypted overlay network. Session affinity keeps a conversation on the replica that already holds its transient prefix state.
- Compute → response. The replica generates the response in GPU and host memory and streams it back through the gateway. When the session's cache is evicted, no trace of the content remains anywhere in the system.
There is no branch of this path where content is written to a disk, a queue, a log, or an analytics system.
Technical controls
| Area | Control |
|---|---|
| Encryption in transit | TLS on the external edge; an encrypted overlay network between the gateway and compute nodes. |
| Data at rest | No customer content at rest, anywhere, ever. Retained operational data (metadata only) is access-restricted; see the retention table. |
| Access control | Single authorized API caller. Production access for personnel is limited, credentialed, and provides no path to customer content: debugging uses synthetic traffic only. |
| Audit & logging | Structured logs record system activity (timestamps, status codes, token counts, latency) and exclude content payloads by construction, not by after-the-fact redaction. |
| Failure handling | Overload is refused with 429 at admission; replica loss is absorbed
by gateway failover. Failure modes are designed to shed load, not spill data. |
Infrastructure & subprocessors
Infer sources GPU compute from infrastructure providers selected for performance and efficiency. Because content is never durably stored, subprocessors host volatile computation only; they never receive a copy of customer content.
Infrastructure subprocessors are currently undergoing compliance evaluation. As engagements are finalized, each provider is assessed for security posture and, where workloads require it, Business Associate Agreement coverage; the resulting subprocessor list will be published on this page and kept current. Workloads subject to regulatory requirements will be placed only on infrastructure that has completed this evaluation.
Shared responsibility
Security of an inference workload is shared between Infer and the customer. The division is:
| Responsibility | Infer | Customer |
|---|---|---|
| Content retention & disposal | Zero retention, enforced architecturally | Retention of prompts/outputs in your own systems |
| Transport security | TLS at our edge; encrypted internal transport | Security of your path to the routing layer |
| Access control | Gateway authentication; no personnel access to content | Credential handling, end-user authentication and authorization |
| Data appropriateness | Publishing accurate handling commitments | Deciding what data may be sent (including regulated data; see HIPAA) |
| Incident handling | Detection, response, and notification for our systems | Response within your systems and to your users |
Third-party attestation
Independent attestation of these controls is part of Infer's compliance program, with SOC 2 as the first planned engagement. Attestation status will be published on this page as milestones complete. Security documentation beyond this overview is in preparation and will be made available to qualifying customers on request.
Reporting a vulnerability
If you believe you have found a security issue in Infer's service, we want to hear about it. Reports are acknowledged promptly, investigated, and answered; please include enough detail to reproduce the issue:
contact@infer.industries