NoisyHate: Hate Speech Detection with Text Perturbations
A NoisyHate (hate speech benchmark) dataset of 1,339 labeled sentences with text perturbations and toxicity scores, collected by NoisyHate Authors via ICWSM 2025. Each record contains sentence text, binary hate label, toxicity scores (toxicity, severe, obscene, identity_attack, insult, threat), identity group indicators, and engagement metrics. The dataset is notable for its benchmark dataset with human-written text perturbations and fine-grained toxicity metrics. Intended for research into hate speech detection, model robustness, adversarial testing, toxicity analysis, and identity-based hate speech research.
Cite this dataset at
noisyhate.echo.directory
Analytics
Topic 0
Topic 1
Topic 2
Topic 3
Topic 4
Topic 5
Topic 6
Topic 7
Topic 8
Topic 9
Topic 10
Topic 11
Show all 15 topics
Topic 12
Topic 13
Topic 14
Computed over the full corpus.
Record
- Catalog key
noisyhate- Authors
- Ye, Yiran; Le, Thai; Lee, Dongwon
- Year
- 2025
- Source paper
- NoisyHate: Mining Online Human-Written Perturbations for Realistic Robustness Benchmarking of Content Moderation Models
- DOI / arXiv
- 10.1609/icwsm.v19i1.35961
- License
- CC BY 4.0 (CC-BY-4.0)
- Records
- 1,339
- Text corpus
- sentence text
- Languages
- en
- Collected
- 2025 (2020s)
- Tasks
- Toxicity detection (regression;
toxicity_toxicity, toxicity_severe_toxicity, toxicity_obscene, toxicity_sexual_explicit, toxicity_identity_attack, toxicity_insult, toxicity_threat) inferred, Hate speech detection inferred
Provenance
- Originally published at
- https://zenodo.org/records/14654784 link checked
- Hosted copy
- Mirrored on echo.directory on 2026-10-09, 411.6 KB across 1 files. Checksums in MANIFEST.json.
Citations
NoisyHate: Mining Online Human-Written Perturbations for Realistic Robustness Benchmarking of Content Moderation Models
Yi-Ran Ye, Thai Le, Dongwon Lee
Journal Article · International Conference on Web and Social Media · 2023
Cited by
- Developing continuous toxicity detection against increasing types of perturbed toxic text
Hankun Kang, Jianhao Chen et al. · Information Processing & Management · 2026 · cited 5× - Breaking Free from Ivory Tower: Evaluating and Enhancing Real-world Chinese Underground Adversarial Jargon Detection
Zhifan Jiang, Ming-Xuan Liu et al. · IEEE Symposium on Security and Privacy · 2026 · cited 2× - Evasion-hate: targeted augmentation for hate speech detection under prompt-induced surface-form shifts
Thanh Le, Thien Khai Tran · Data Technologies and Applications · 2026 · cited 0× - Requirements engineering for countering hostile narratives: a vision for socio-technical intervention systems beyond detection
Sofia Almpani, P. Loucopoulos et al. · Requirements Engineering · 2026 · cited 0× - ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbations
Hankun Kang, Xin Miao et al. · The Web Conference · 2026 · cited 0×
Citation data from Semantic Scholar, retrieved 2026-10-22.
How to cite
BibTeX
@misc{echo2026,
title = {ECHO: Exploratory Compendium of Humanity Online},
author = {Switzer, Braden and Zafarani, Reza},
year = {2026},
url = {https://echo.directory},
note = {Syracuse University}
}
@article{Ye_2025,
title = {NoisyHate: Mining Online Human-Written Perturbations for Realistic Robustness Benchmarking of Content Moderation Models},
volume = {19},
issn = {2162-3449},
url = {http://dx.doi.org/10.1609/icwsm.v19i1.35961},
doi = {10.1609/icwsm.v19i1.35961},
journal = {Proceedings of the International AAAI Conference on Web and Social Media},
publisher = {Association for the Advancement of Artificial Intelligence (AAAI)},
author = {Ye, Yiran and Le, Thai and Lee, Dongwon},
year = {2025},
month = June,
pages = {2603–2612}
} APA
Switzer, B., & Zafarani, R. (2026). ECHO: Exploratory Compendium of Humanity Online [Data repository]. Syracuse University. https://echo.directory
Ye, Y., Le, T., & Lee, D. (2025). NoisyHate: Hate Speech Detection with Text Perturbations [Data set]. https://doi.org/10.1609/icwsm.v19i1.35961
MLA
Switzer, Braden, and Reza Zafarani. ECHO: Exploratory Compendium of Humanity Online. Syracuse University, 2026, echo.directory.
Ye, Yiran, et al. NoisyHate: Hate Speech Detection with Text Perturbations. 2025, doi.org/10.1609/icwsm.v19i1.35961.