Compare tagging and dependency parsing systems across languages, training setups and written or spoken test data.
Opened from the file system. The table works, but the browser will
not load a run's detailed analysis this way. Serve the tables folder
instead: cd tables && python3 -m http.server 8000, then open
http://localhost:8000/am_benchmark_v2/.
Detailed analysis
Part-of-speech tagging, lemmatisation and morphological features, compared with the manually annotated test set.
For each UPOS tag, Accuracy is the percentage of its words the system tagged correctly; Occurrences is how often the tag appears in the test set. Most frequent tags first.
Every word the system annotated differently from the test set, counted separately for part of speech, lemmas and features.
Dependency parsing: each word's head and its Universal Dependencies relation, compared with the manually annotated test set.
For each relation, LAS is the percentage of its words with both the correct head and the correct relation; Occurrences is how often the relation appears in the test set. Most frequent relations first.
Every word with a wrong head, a wrong relation or both falls into exactly one of three types, A, B or C, so the three numbers below add up to the total.
Everything the CoNLL 2018 evaluation script reports for this run. Precision is the share of the system's annotations that are correct, recall the share of the test set's annotations the system reproduced, and F1 their harmonic mean. F1 is the score in the overview table and the summary above. The per-tag and per-relation tables under Morphology and Syntax use a simpler measure: the percentage of a label's occurrences in the test set that the system got right.
| Item | Value |
|---|
The overview table, the summary above and All evaluation metrics read their scores from this one file.
| Item | Value |
|---|