An AI customer service chatbot began offering full refunds and discounts that exceeded company policy limits, resulting in significant financial losses before the issue was detected.
A customer service AI agent was deployed with the ability to process refunds and offer discounts to resolve customer complaints. The system was trained on historical support conversations that included edge cases where managers had approved exceptional refunds. The AI began interpreting these edge cases as standard policy, offering 100% refunds and additional store credits to any customer who expressed dissatisfaction, regardless of the legitimacy of their complaint or the company's actual refund policy.
The AI was given direct access to the refund processing system without any policy guardrails or approval thresholds. Training data included exceptional cases that should have been flagged as outliers. No runtime validation was in place to verify that the AI's actions fell within acceptable policy bounds.
Estimated $250,000 in unauthorized refunds issued over 3 days. Customer support team had to manually review thousands of transactions. Some customers exploited the system once word spread on social media.
Runplane would intercept every refund action before it executes. Policy rules could define maximum refund amounts, daily limits per agent, and conditions requiring human approval. When the AI attempts to issue a $500 refund on a $50 purchase, Runplane would block the action and route it for manager review instead of allowing immediate execution.