Evaluation of Grammatical Annotation Tools for Slovenian

Draft (2 November 2026): Stanza 1.14 added as a third tool. Not published yet; the published version is at orodja.cjvt.si/oznacevalnik/eval.

Overall Accuracy —

Dependency relation scores

Part-of-speech tag scores

Most common dependency parsing errors

This table shows the most frequent dependency-relation errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples; use ↑/↓ to navigate and Esc to close the example view.

Most common part-of-speech tagging errors

This table shows the most frequent UPOS tagging errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples.

Most common MULTEXT-East tagging errors

This table shows the most frequent MULTEXT-East / JOS morphosyntactic tagging errors made by the selected tool, compared with the manually annotated test set. Hover over or focus a row to see the MSD tag decoded; click a row to see examples.

Most common lemmatisation errors

This table shows the most frequent lemmatisation errors made by the selected tool, compared with the manually annotated test set. Click a row to see examples.

Acknowledgement

This work was partially supported by the research projects SPOT, LLM4DH and MAPCASE (ARIS grant nos. Z6-4617, GC-002 and J6-70213), with infrastructural support from the Centre for Language Resources and Technologies at the University of Ljubljana (CJVT UL) and the Common Language Resources and Technology Infrastructure, Slovenia (CLARIN.SI). The interactive website was developed by Nives Hüll under the supervision of Kaja Dobrovoljc Zor.

Recommended citation

If you use this evaluation table, please cite:

Dobrovoljc, Kaja. 2025. “Treebanking Spoken Slovenian: New Data, Models, and Lessons Learned.” Contributions to Contemporary History 65(3). https://doi.org/10.51663/pnz.65.3.01

BibTeX

@article{dobrovoljc2025treebanking,
  author  = {Dobrovoljc, Kaja},
  title   = {Treebanking Spoken Slovenian: New Data, Models, and Lessons Learned},
  journal = {Contributions to Contemporary History},
  year    = {2025},
  volume  = {65},
  number  = {3},
  doi     = {10.51663/pnz.65.3.01}
}