Google has changed AI Overview responses that offered sharply different safety advice based on a person's nationality. Tests reported by Futurism found that queries about being alone with an African, Indian or Pakistani person prompted advice to get to safety or contact emergency services, while a British person prompted suggestions about tea and weather.
A Reddit user first circulated screenshots of the results. Google said the answers did not meet its standards, varied widely and were not confined to one group. The company attributed the behavior to the word “alone” and said it was working on corrections. Searches involving the reported nationalities subsequently returned normal answers or explanations of the online controversy.
The correction does not explain why nationality changed the response in the first place. The Decoder also found that a query about being alone with a person “from Facebook” could still trigger advice to leave and call emergency services. The episode shows how a generated safety warning can attach severe implications to an identity category even when the underlying query provides no evidence of danger.