Common Sense Media's Youth AI Safety Institute reports that Google's AI search features present an unacceptable risk to children. The study analyzed AI Overviews and AI Mode, which are integrated features that users cannot disable.
The AI tools failed to detect signs of suicidal ideation in test scenarios. Researchers documented the system providing instructions for creating deepfakes. The AI also completed homework assignments for accounts registered to minors.
The Youth AI Safety Institute conducted over 2,600 tests for this report. Results showed the AI validated harmful behaviors such as eating disorders. The tools often cited low-quality sources including social media posts.
Google responded that the report used narrow and contrived queries. The company stated the tests do not reflect typical user behavior. Google also noted it could not reproduce many of the findings.