ShieldGemma: Generative AI Content Moderation Based on Gemma
Introduces ShieldGemma, a suite of content-moderation models built on Gemma 2 (roughly 2B to 27B parameters) that classify safety risks across four harm types in both user inputs and model outputs. The paper also describes a synthetic data-curation pipeline for building moderation training sets. The authors report improved area-under-PR-curve over Llama Guard and WildGuard on their evaluations.
Publisher
Published
31 Jul 2024
Added
3 months ago
DOI
—
Key Findings
- A family of moderation classifiers at multiple sizes (about 2B-27B) built on Gemma 2, covering four harm types for both inputs and outputs
- Reports area-under-PR-curve gains of roughly 10.8% over Llama Guard and 4.3% over WildGuard on the authors' benchmarks
- Describes an LLM-based data-curation pipeline for generating moderation training data
Methodology Notes
Technical report (arXiv) from the model's developers; benchmark comparisons are the authors' own runs, not independent evaluations. A later ShieldGemma 2 covers image moderation.
Sources
arXiv preprint(opens in a new tab) (primary)
Archived snapshot (Wayback Machine)(opens in a new tab) — preserved against link rot
Authors
Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, Oscar Wahltinez
Tags
Cite This
APA
Wenjun Zeng et al. (2024). ShieldGemma: Generative AI Content Moderation Based on Gemma. Google. https://arxiv.org/abs/2407.21772