The company describes Jay Guard as a security layer between an application and an LLM, checking attacks, content, and personal data.
The team evaluated GPT-OSS Safeguard 20B, Llama Guard 4 12B, and a fine-tuned 184M-parameter DeBERTa model. Its validation set contained 436 Russian- and English-language jailbreak and injection examples plus 121 benign requests from logs. With default policies in zero-shot use, GPT-OSS Safeguard recorded F1 of 0.57 and 38% false positives on benign requests; Llama Guard 4 recorded F1 of 0.45 and 45% false positives. The fine-tuned DeBERTa recorded F1 of 0.93 and 1.7% false positives.
Just AI distinguishes a guard model, which classifies text, from a guardrails system that combines specialised detectors, policies, thresholds, and enforcement points. It also notes that text detection does not address supply-chain threats such as malicious MCP servers or poisoned weights.
The source does not establish that these results generalise beyond its validation data, default configurations, or operational policies. https://habr.com/ru/companies/just_ai/articles/1084804