As large enterprises build generative AI into business workflows, security leaders need to understand not only which controls are required to secure AI capabilities, but where and how those controls should operate within the broader enterprise architecture. This is particularly important because generative AI introduces a fundamentally different security challenge: probabilistic systems do not always produce the same behavior under the same conditions, and their behavior can be influenced by natural-language instructions, retrieved data, application context, and increasingly, the tools and systems they are permitted to access.
Traditional security architecture is built largely around deterministic controls, where an identity is authenticated, a policy is evaluated, and an action is either allowed or denied. With AI, organizations must secure a system whose behavior cannot always be reduced to a predictable set of paths. That does not make established security principles obsolete, instead it makes defense in depth even more important. Identity, authorization, data protection, application security, monitoring, and governance remain essential, but organizations also need controls designed for the AI interaction itself. OCI Generative AI guardrails is one of those controls, providing an additional policy and detection layer around what enters and leaves a generative AI system. Security therefore needs to exist around the model as well as within it.
The foundations remain, but AI introduces new control requirements
Generative AI does not invalidate the foundations of enterprise security. If an AI application can retrieve data that a user should not be authorized to access, the underlying access-control problem still needs to be addressed. What changes is that these established controls must now operate alongside controls designed for the probabilistic nature of AI.
One example is the use of classifiers around the model interaction. At a high level, a classifier evaluates content and determines whether it exhibits a particular characteristic or risk. Instead of relying exclusively on the generative model to recognize that a request is malicious or that its response contains sensitive information, a separate control can inspect the interaction and return a signal that the application can use to determine what happens next.
An input classifier can therefore sit in front of the model and evaluate a prompt before inference. An output classifier can perform a similar evaluation after the model generates a response. Teams must intentionally integrate or configure guardrails as part of the application or endpoint architecture and should not assume that every model invocation is automatically inspected by the same guardrail policy layer. Together, they create an additional control boundary around the model:
User or application → Input evaluation → Model → Output evaluation → Application or user
Classification and enforcement can also serve distinct functions. A classifier can identify a risk and provide a signal, while the surrounding application determines what that signal means based on organizational policy. Depending on the use case, the application might block the interaction, warn the user, redact information, record the event, require additional review, or use the result as telemetry for security monitoring. Actions such as redaction, blocking, escalation, and retention depend on application logic or endpoint configuration rather than classification alone. For on-demand models, the ApplyGuardrails API evaluates supplied content and returns results for the application to handle. On supported dedicated AI cluster endpoints, guardrails can run in Inform mode or Block mode for inputs and outputs.
OCI Generative AI Guardrails applies this pattern to several risks relevant to enterprise AI:
- Content moderation evaluates inputs and outputs for harmful or policy-restricted material.
- Prompt injection detection looks for malicious or unintended instructions intended to override system behavior, expose hidden instructions, or manipulate tool and data access, including instructions embedded within retrieved content.
- PII detection identifies sensitive personal information such as names, email addresses, and telephone numbers.
This additional layer matters because AI applications increasingly process content that cannot automatically be trusted. A prompt may originate with an employee, customer, external application, retrieved document, or other data source. In an agentic architecture, that information may ultimately influence which tools are called and what actions are taken. Prompt injection makes this particularly important because malicious instructions can be introduced indirectly through content the application retrieves rather than being explicitly entered by the user. Applications should therefore deliberately pass relevant user input, retrieved context, and tool output through the appropriate inspection path.
Guardrails give architects a dedicated control point for inspecting these interactions before they reach the model or are returned to the application. Because guardrails are model- and classifier-based controls, teams should account for false positives and false negatives rather than treating detection as a guarantee.
Where guardrails fit in the architecture
As AI applications move beyond simple prompts and responses, the content reaching a model may come from users, conversation history, retrieved documents, databases, or tool responses. A legitimate user can therefore make a legitimate request while retrieved content introduces instructions intended to manipulate the model.
This is where the placement of controls matters. Identity establishes who is making the request, authorization governs what data and systems that identity can reach, and least privilege limits what an application or agent can ultimately do. OCI Generative AI guardrails addresses a different problem: the content moving through the AI interaction itself.
Identity and Authorization → Application → Input and Context Inspection → Model → Output Inspection → Authorized Actions
These controls work together. Prompt injection detection adds protection against attempts to manipulate model behavior, while least privilege limits the resources and actions available to an application or agent. PII detection provides an additional inspection point for sensitive information, while access controls govern who can retrieve that information in the first place. And when an agent can take consequential actions, authorization controls can govern whether those actions are permitted.
For CISOs, guardrails add specific controls for the prompts and responses that move through AI applications, complementing existing controls such as identity, authorization, and least privilege. Operationally, organizations should test guardrails against representative prompts and retrieved content, monitor outcomes, review blocked and informed events, and pin or review guardrail versions where applicable.