AI Infrastructure Requirements for SOC 2 Compliance

SOC 2 was designed around three implicit assumptions: controls are static, systems behave deterministically, and humans mediate access to sensitive operations. Agentic AI violates all three simultaneously, and the auditors who haven't caught up to that yet will.

A traditional web application receives a request, processes it through known code paths, and returns a response. The access control logic is written once and reviewed once. Predictability is what makes conventional SOC 2 audits tractable in the first place.

An agent does none of that. It generates and executes code at runtime. It modifies its own behavior based on retrieved context, tool outputs, and intermediate reasoning steps. It makes dozens of decisions per request without a human authorizing each one. A single customer support interaction in an agentic system might touch a planner, a retriever, a policy checker, a billing tool, a summarization model, and a human-review queue. Each transition is a distinct control boundary. SOC 2 asks whether access control, change management, monitoring, vendor risk, and incident response hold across the entire chain, not just at the front door.

The cost exposure compounds the compliance exposure in a way that's easy to miss until you're already in trouble. Agents run like a meter: each step generates a separate charge, and the agent decides how many steps to take. An uncontrolled agentic loop can exhaust a budget and produce unreviewed outputs simultaneously, which is both a processing integrity failure and a monitoring gap.

The real compliance question agentic systems force is this: at what layer do you enforce identity, policy, content filtering, spend limits, and logging when there is no single human-triggered request to attach controls to? The control plane has to sit below the agent, not inside it. That layer is the gateway.

The Gateway Layer as a Unified Control Plane for SOC 2 Evidence

Gartner defines an AI gateway as a technology that acts as an intermediary between applications and AI services, providing a central point for security, governance, and observability. The market went from a fraction of its current size in 2023 to a multibillion-dollar segment by 2024, which tells you how quickly enterprises recognized the control problem.

When every model call passes through one layer, that layer becomes the single most sensitive service in the stack and, usefully, the single most auditable one. No application communicates directly with an external model provider. The gateway holds every provider credential, compressing the secrets problem from dozens of scattered keys to one guarded boundary. It is the only component with egress capability to external APIs.

Each Trust Service Criterion maps to a gateway control. Security (CC6) is addressed through RBAC at the API key level: every request attributed to a team, project, or individual, with no shared service accounts. Availability is addressed through fallback routing and provider failover, so when one provider degrades, traffic reroutes without application changes. Confidentiality is addressed through PII redaction applied before requests reach any provider, and output scanning before logging. Processing Integrity is addressed through policy enforcement on which models may be called for which workloads. Privacy is addressed through data residency enforcement, ensuring regulated data is never routed to providers outside permitted jurisdictions.

SOC 2 Type II requires continuous evidence across the full audit period, typically six to twelve months. Logs must exist from day one; they cannot be reconstructed after an auditor asks for them. Structured logs covering user identity, model, token counts, latency, cost, and policy decisions satisfy CC7 monitoring requirements and, when exported to OpenTelemetry-compatible SIEMs like Grafana, Datadog, or Splunk, stay under the customer's control throughout the audit window.

Without a gateway, the failure modes are predictable and well-documented: keys embedded in source code, no per-request attribution, no enforceable rate limits, PII reaching providers unguarded. Each one is a CC6 or CC7 finding waiting to surface.

Access Control and Identity Requirements That Auditors Check at the API Layer

CC6 requires that logical access is restricted to authorized users and that access is attributable. In AI systems, "user" means every application, service account, and human engineer making a model call. That scope is wider than most engineering teams initially recognize, and it tends to surface as a surprise during audit prep.

The shared-key problem is the most common finding in AI deployments. A single API key used across three teams cannot tell an auditor which team made which call. That gap fails CC6's attribution requirements, and it appears in audit after audit. Roughly two-thirds of B2B buyers now require a SOC 2 report before engaging a vendor, which means this particular gap is increasingly a commercial blocker, not a compliance technicality you can defer.

What auditors want to see: unique credentials per team, project, or workload (not necessarily per developer, but per logical boundary); RBAC that restricts which models and providers each credential may call; rotation logs showing when keys were rotated, by whom, and what triggered it; offboarding evidence showing that when a team member leaves, their access is revoked and the event is recorded. SSO integration connects AI access to the organization's identity provider so that provisioning and deprovisioning become auditable in one place rather than scattered across individual team practices.

When each team builds its own access logic at the application layer, enforcement is inconsistent, not auditable across teams, and invisible to the security function. When access control lives at the gateway, one policy engine produces one audit log with consistent enforcement regardless of which application or programming language originated the request. That consistency is what makes an audit go smoothly rather than sideways.

Agentic systems complicate this further. The identity making a request may be a planner component, not a human. RBAC must be defined per agent role, not just per human user, and the audit log must capture the agent's identity alongside the initiating human session. Auditors reviewing agentic architectures now expect this level of granularity, and the ones who don't yet will soon.

PII Redaction, Subprocessor Risk, and What "Data Never Reaches the Provider" Means in Practice

The core privacy risk in LLM deployments is straightforward enough to state plainly: an application passes a prompt containing customer PII to an external model provider, the provider logs it, and the organization has no record of what was sent or that it was sent at all. The Confidentiality criterion requires that data classified as confidential is protected from unauthorized disclosure. Disclosure to a subprocessor beyond what is contractually and operationally necessary qualifies.

Every LLM provider called in production is a subprocessor. Auditors require a vendor risk assessment for each one, covering whether the provider holds their own SOC 2 or equivalent, what their data retention and logging policies are, and whether data processing agreements are in place. In architectures calling multiple providers, maintaining individual DPAs and assessments for every provider is operationally expensive; the gateway consolidates that trust boundary, reducing a sprawling vendor risk surface to a single managed relationship.

PII redaction at the gateway layer means redaction fires before the request exits to any provider. The provider never receives the sensitive token. Audit logs should record that redaction fired on a given request, what category of PII was detected, and which rule applied. Output scanning extends this to model responses before they are logged, preventing sensitive data from entering the audit record itself.

Zero data retention agreements with providers add a contractual layer on top of technical controls. Some providers commit that request data is not logged or used for training; auditors want to see these agreements in place for workloads handling regulated data, and they want the workload-to-agreement mapping documented, not gestured at.

Data residency enforcement closes what remains. For GDPR-adjacent requirements or regulated industries, routing rules tied to data classification must determine which providers and regions may receive which data. That logic cannot be left to individual application developers to implement consistently. Sensitive customer data, whether PII, PCI, or PHI, should never reach model providers unguarded, and redaction must be reviewable and enforceable, not aspirational.

Availability and Processing Integrity Controls for Production AI Systems

The Availability criterion requires that systems are available for operation as committed. In AI infrastructure, the primary availability risks are provider outages, rate limits, and model deprecations. All three occur within audit periods, and all three are well-documented in the operational history of major LLM providers.

A provider incident that cascades into an application incident is a SOC 2 availability event, full stop. Without fallback routing, there is no technical distinction between "the provider is down" and "our product is down." Fallback routing as an availability control means routing rules that automatically redirect traffic to a secondary provider when the primary is unhealthy. Evidence of a tested failover procedure satisfies CC7 monitoring requirements more convincingly than a policy document alone, and that distinction matters when an auditor is deciding whether to qualify an opinion. Model deprecation risk extends this: providers retire models with varying notice periods; gateway-level model abstraction means the application code does not change when the underlying model is swapped, which preserves both availability and operational stability.

Processing Integrity requires that system processing is complete, valid, accurate, timely, and authorized. Drift is the integrity risk LLMs introduce that has no real analogue in traditional systems. A model updated by a provider mid-audit-period may produce materially different outputs for identical inputs. SOC 2 requires that change be monitored and documented, not discovered through customer complaints after the fact.

Drift monitoring means continuous scoring of production traffic against a baseline. Output quality changes that surface as customer tickets are a lagging indicator; structured evals running against production traffic are the leading one, and they're the difference between catching something and explaining it. CC8 governs change management and requires that any change to routing rules, model versions, or provider configuration go through a controlled process with approval and logging. In LLM infrastructure, that process needs to account for provider-initiated changes that occur without the customer's direct action.

Rate limits represent an availability risk distinct from outages and one that's easier to overlook. A single runaway agent or misconfigured workload can exhaust provider rate limits for the entire organization. Gateway-level rate limiting and spend caps prevent one workload from degrading availability for others, and the enforcement events are logged, producing the monitoring evidence CC7 requires.

AI Spend Visibility as a SOC 2 Control Requirement, Not Just a Finance Problem

Token spend is not a stable line item, and treating it as one is a governance failure before it becomes a compliance problem. Per Ramp data, token usage grew substantially from January 2025 through April 2026 even as per-token prices declined, with dollar spend still increasing and monthly swings remaining large. Among companies tracked in April 2026, median monthly AI spend per employee was $46, but the middle half of companies ranged from $3 to $352 per employee. That variance does not reflect seasonal patterns. It reflects how differently organizations actually govern usage.

The compliance angle on spend data gets overlooked because finance tends to own the budget conversation and security tends to own the audit conversation, and the two don't talk as often as they should. Granular, real-time spend visibility by team, project, key, model, and provider is the evidence base for CC7 system monitoring and CC5 control activities. An auditor reviewing CC7 wants to see that anomalous usage was detected and investigated. A gateway that enforces hard budget limits and logs enforcement events produces that evidence automatically, as a byproduct of normal operation, not a separate compliance project.

End-of-month provider invoices do not satisfy Type II continuous monitoring requirements. They show what was spent; they do not show when an anomaly was detected, what triggered the alert, or what remediation occurred. Per-request cost attribution logged at the gateway does all of that.

The allocation problem is structural in most organizations. AI costs appear in cloud bills under compute, storage, or data processing categories, not as identifiable AI line items. Cost-to-workload attribution is nearly impossible without a layer that tags every request at the moment it is made.

Observability platforms report on spend after it occurs. A traditional alert notifies a team of an overrun but contains no attribution to the model or prompt that caused it and no mechanism to stop the next one. Gateway-level enforcement stops the request before it reaches the provider. The policy decision is logged, and that log is what a Type II audit is actually looking for: not just that controls exist, but that they operated, continuously, across the entire audit period.