Most AI security attention goes to the model. Most incidents happen around it: the document that contains an instruction, the tool that can delete a record, the prompt that includes a customer’s personal data.
A useful AI security review looks at the system, not just the model.
Prompt injection
Prompt injection is an instruction hidden inside content the model reads. A support ticket, a PDF or a web page can contain text that tells the model to ignore its rules, reveal its prompt or call a tool.
The defence is not a better prompt. Treat every retrieved document as untrusted input. Separate instructions from data, limit what the model can do with retrieved content, and require confirmation before any action that changes a record.
Data leaving through tools
An AI feature often has tools: search, email, CRM lookup, file access. Each tool is a path for data to leave the intended boundary. A summarizer with access to the whole drive can include an HR file in a sales summary.
Scope tools to the task. Use service accounts with the minimum permissions, log every tool call, and keep sensitive systems out of the default tool set.
Over-permissioned agents
An agent that can read invoices should not be able to approve payments. An agent that drafts customer replies should not be able to send them without review. Permission should follow the action, not the convenience of the demo.
Break risky actions into two steps: prepare and approve. The agent prepares; a person approves. That single pattern prevents most irreversible mistakes.
Vendor retention and training
Ask where prompts and outputs are stored, how long they are retained, whether they are used to improve the model, and which subprocessors are involved. “Enterprise” is not a retention policy.
For personal data, the answer belongs in the DPDP records. If the vendor cannot explain retention and deletion, the feature is not ready for customer data.
Outputs in decisions
A model output used to rank a candidate, approve a loan or set a price is a decision input. It needs a review path, a record of the model and prompt version, and a way for a person to challenge the result.
Do not put a model score in a decision without an owner. The owner is accountable for the outcome, not the vendor.
Controls that fit a small team
Start with an inventory of AI features and the data each one touches. Add a review step for any action that changes money, access or customer records. Log prompts and tool calls for a defined period. Test one prompt-injection case and one data-leak case before launch.
Review vendor terms at renewal, not only at signing. Retention and model-training terms change.
A pre-launch AI security review
Run the review before the feature reaches customers. List every model, tool, data source and vendor in the flow. For each one, write what it can read, what it can change and who can see the output.
Test three cases. Put an instruction inside a retrieved document and confirm the model ignores it. Ask the model to reveal its system prompt and confirm it refuses. Try to access a record outside the user’s permission and confirm the tool blocks it.
Check the logs. Can you reconstruct which document was retrieved, which tool was called and which model version answered? If not, the feature is difficult to investigate after an incident.
Set a review date. Model, prompt and tool changes should trigger a fresh review, not wait for the annual security cycle.
FAQ
Is prompt injection a real risk for internal tools?
Yes. Internal documents and emails can contain hostile text too. The risk is lower when the tool is read-only, which is why read-only pilots are safer.
Do we need a separate AI security policy?
A short addendum to the existing information security policy is usually enough: approved tools, data classes, review steps and incident reporting.
Can we use AI with customer data?
Yes, with a defined purpose, minimum necessary data, a processor agreement and retention rules. The DPDP documentation should name the AI vendor.
For a review, see cyber security services cost and DPDP Act compliance cost. Managed teams can start with managed IT services.
