Welo Data, a division of Welocalize and the leader in high-quality multilingual AI training data, has released new research showing that safety protections in leading large language models (LLMs) do not reliably transfer across languages. The study evaluated 10 leading models using 210,000 model-prompt pairs across 79 languages and safety categories such as Hate and Discrimination, Self-harm and Suicide, Violence and Threats, and Misinformation and Disinformation. Unlike most existing safety evaluations, which are conducted primarily in English, this study measured cross-lingual safety gaps, exposing multilingual safety as a major challenge. The results reveal a consistent pattern: safety alignment is strongest in English but deteriorates across models in low-resource, or digitally under-represented, languages. 

Even in areas where guardrails are effective in English, like Hate and Discrimination and Self-harm and Suicide, the same prompts in low-resource languages can “jailbreak” those protections. A safe response in English often becomes an unsafe response in another language, exposing how safety reduces as linguistic representation declines. This erosion exposes a global security gap: even strong English guardrails can be bypassed simply by translating harmful prompts into another language using readily available translation tools. It also creates a safety blind spot for populations that primarily interact with LLMs in non-English languages, leaving them with weaker protection against harmful content. For global developers and organizations deploying LLMs across regions, this is not just a technical limitation; it’s also a safety liability. 

“As expected, LLM performance and safety robustness are greater in English given its dominance in the training data,” said Dr. David Harper, Data Scientist at Welo Data and lead author of the study. “If policy enforcement and evaluation frameworks are anchored only in English, we risk deploying systems that behave safely in one environment while enabling harm in another. And because online tools can translate content across languages instantly, weaknesses in low-resource languages become entry points for unsafe outputs. Multilingual safety must be treated as a core requirement for responsible global deployment.”

Key observations from the study include:

• Models vary in their safety behavior in English. Safety performance in English varies by model, indicating that alignment mechanisms operate differently across harm domains. In our sample, the categories of Hate and Discrimination and Self-harm and Suicide are the most consistently protected across models.
• Prompting in low resource languages can increase the likelihood of a harmful response by 4-5X.  Even the categories with the strongest guardrails in English degrade sharply in low-resource languages. For instance, unsafe response rates in the Hate and Discrimination category climbed from below 10% in English to 40–50% in several low-resource languages.
• Models vary in their multilingual safety. Some models maintain relatively consistent safety performance from English to low-resource languages, while others show sharper degradation. These differences suggest that safety mechanisms do not generalize equally well across models.
• The gap is systematic, not incidental. Safety degradation correlates with language family. Languages from the Niger-Congo and Nilo-Saharan families exhibit the greatest increases in unsafe completions.

For more information, visit welodata.ai.