It has been revealed that legal documents submitted to a court contained "hidden AI instructions" intentionally embedded to manipulate automated document review systems. This is a type of prompt injection designed to force the system to read text invisible to humans and alter its processing behavior.
The technique identified involves inserting invisible instructions into the background of documents used in legal proceedings, prompting the AI model to produce specific outputs or judgments. Through document layout adjustments, font settings, and specific metadata manipulation, the method aims to intervene in the AI system's decision-making process without being detected by human eyes.
This incident exploits a vulnerability where AI models, when analyzing documents, process not only visual information but also hidden text contained within PDF and text data structures. It suggests that automated review systems increasingly being adopted by courts and other institutions lack sufficient defenses against malicious prompts.
This issue has highlighted the urgent need to review security standards for automation technology in legal processes. Moving forward, it will be essential to strengthen input filtering on the system side and establish processes to verify the integrity of documents analyzed by AI. This case serves as a renewed reminder of the critical importance of input data censorship and verification in fields where AI implementation is advancing.