Evaluation Benchmark

A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP
This paper presents a Japanese phamaceutical-specific langauge model, JPharmatron, along with a benchmark suite consisting three new benchmarks: YakugakuQA (Japanese national pharmacist licensing exams); NayoseQA (cross-lingual synonym and terminology normalization); and SogoCheck (consistency reasoning between paired statements.)
Issey Sukeda, Takuro Fujii, Kosei Buma, Shunsuke Sasaki, Shinnosuke Ono
Cite PDF Code Dataset DOI
A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP