A bank wants to deploy an always-on AI agent for its procurement team. Design the rollout.
Think first
Identity, approvals, audit and evaluation, before capabilities.
Strong answer
Define the job narrowly first: e.g. process purchase requests under a threshold against policy, chase missing quotes, prepare POs for approval. Give the agent its own non-human identity with least-privilege access to the procurement system, read-only to the vendor master, and no payment rights. Write a tiered approval policy: reading and drafting allowed, external emails and PO creation require a named approver, anything touching vendor bank details is never automated. Stream every action with the agent identity, approver and outcome to the bank's SIEM. Build an eval set from a few hundred historical requests and require it to match human outcomes before moving from shadow mode to supervised mode. Run a 90-day pilot with an owner in procurement and a platform-team owner. Expect model risk management and internal audit to review it, so the evidence pack (policy, eval results, logs) is a deliverable.
Weak-answer indicators
- Agent uses a human's account
- No approval tiers
- No eval before go-live
- Forgets audit and model risk review
Likely follow-up questions
- The vendor updates the underlying model mid-pilot. What do you do?
- Approval rates are 99.5%. Good or bad?