Unsupervised and Very Low Resource MT

There is no machine translation available for most of the ~7000 languages spoken on the planet Earth. In the unsupervised and very low resource translation task, we collaborate with the local communities in providing resources and developing MT systems for local minority languages. Like last year, the task will include translation of Upper Sorbian, a minority Slavic language spoken in Germany. This year, we added a translation of closely related Lower Sorbian and Chuvash, a minority Turcic language spoken in south Russia. This year, the tasks are: Unsupervised Machine Translation: German to Lower Sorbian. Lower Sorbian to German. Very Low Resource Supervised Machine Translation: German to Upper Sorbian. Upper Sorbian to German. Low Resource Supervised Machine Translation: Russian to Chuvash. Chuvash to Russian.

Register your team Create submission Competition updates

Leaderboard of Unsupervised and Very Low Resource MT

Chuvash to Russian test set (chv-ru)

# Name BLEU chrF Date
1 Anonymous submission #66 23.4 0.476 July 23, 2021, 2:24 p.m.
2 Anonymous submission #55 23.4 0.476 July 23, 2021, noon
3 Anonymous submission #44 22.0 0.463 July 23, 2021, 7:31 a.m.
4 Anonymous submission #49 21.3 0.460 July 23, 2021, 7:53 a.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

German to Lower Sorbian test set (de-dsb)

# Name BLEU chrF Date
1 Anonymous submission #72 29.9 0.599 July 23, 2021, 4:24 p.m.
2 Anonymous submission #67 29.9 0.599 July 23, 2021, 2:47 p.m.
3 Anonymous submission #34 27.5 0.579 July 22, 2021, 9:34 p.m.
4 Anonymous submission #33 27.4 0.578 July 22, 2021, 9:34 p.m.
5 Anonymous submission #10 24.1 0.542 July 21, 2021, 2:48 p.m.
6 Anonymous submission #11 22.7 0.537 July 21, 2021, 2:55 p.m.
7 Anonymous submission #25 22.4 0.528 July 22, 2021, 3:06 p.m.
8 Anonymous submission #60 8.0 0.321 July 23, 2021, 1:07 p.m.
9 Anonymous submission #12 6.8 0.409 July 21, 2021, 3:13 p.m.
10 Anonymous submission #24 6.8 0.409 July 22, 2021, 2:57 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

German to Upper Sorbian test set (de-hsb)

# Name BLEU chrF Date
1 Anonymous submission #17 66.3 0.837 July 21, 2021, 4:53 p.m.
2 Anonymous submission #28 66.3 0.836 July 22, 2021, 8:50 p.m.
3 Anonymous submission #69 66.2 0.837 July 23, 2021, 4:07 p.m.
4 Anonymous submission #64 65.9 0.833 July 23, 2021, 2:12 p.m.
5 Anonymous submission #62 65.8 0.833 July 23, 2021, 1:24 p.m.
6 Anonymous submission #42 65.8 0.833 July 23, 2021, 7:20 a.m.
7 Anonymous submission #48 65.6 0.832 July 23, 2021, 7:47 a.m.
8 Anonymous submission #57 65.5 0.831 July 23, 2021, 12:52 p.m.
9 Anonymous submission #56 65.5 0.831 July 23, 2021, 12:51 p.m.
10 Anonymous submission #30 63.3 0.819 July 22, 2021, 9:30 p.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

Lower Sorbian to German test set (dsb-de)

# Name BLEU chrF Date
1 Anonymous submission #71 33.5 0.638 July 23, 2021, 4:23 p.m.
2 Anonymous submission #68 33.5 0.638 July 23, 2021, 3:01 p.m.
3 Anonymous submission #35 33.3 0.620 July 22, 2021, 9:34 p.m.
4 Anonymous submission #27 33.3 0.637 July 22, 2021, 5:37 p.m.
5 Anonymous submission #36 33.2 0.618 July 22, 2021, 9:35 p.m.
6 Anonymous submission #21 32.4 0.622 July 22, 2021, 11:58 a.m.
7 Anonymous submission #20 31.6 0.612 July 22, 2021, 11:55 a.m.
8 Anonymous submission #26 18.7 0.498 July 22, 2021, 3:08 p.m.
9 Anonymous submission #18 18.3 0.493 July 21, 2021, 5:01 p.m.
10 Anonymous submission #51 5.9 0.316 July 23, 2021, 11 a.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

Russian to Chuvash test set (ru-chv)

# Name BLEU chrF Date
1 Anonymous submission #54 22.1 0.513 July 23, 2021, 11:24 a.m.
2 Anonymous submission #53 21.8 0.512 July 23, 2021, 11:19 a.m.
3 Anonymous submission #43 20.9 0.501 July 23, 2021, 7:30 a.m.
4 Anonymous submission #45 20.7 0.502 July 23, 2021, 7:31 a.m.
5 Anonymous submission #50 19.4 0.506 July 23, 2021, 8:47 a.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.

Upper Sorbian to German test set (hsb-de)

# Name BLEU chrF Date
1 Anonymous submission #65 67.7 0.834 July 23, 2021, 2:20 p.m.
2 Anonymous submission #41 67.6 0.834 July 23, 2021, 6:26 a.m.
3 Anonymous submission #40 67.6 0.834 July 23, 2021, 6:24 a.m.
4 Anonymous submission #39 67.6 0.834 July 23, 2021, 3:24 a.m.
5 Anonymous submission #38 67.6 0.834 July 23, 2021, 3:16 a.m.
6 Anonymous submission #70 67.3 0.836 July 23, 2021, 4:08 p.m.
7 Anonymous submission #14 67.2 0.836 July 21, 2021, 3:41 p.m.
8 Anonymous submission #29 67.2 0.836 July 22, 2021, 8:54 p.m.
9 Anonymous submission #19 67.0 0.833 July 22, 2021, 9:02 a.m.
10 Anonymous submission #47 64.3 0.819 July 23, 2021, 7:33 a.m.
BLEU and ChrF are sacreBLEU scores. Systems in bold face are your submissions. We only display the top-10 submissions per language pair. Submission validation errors denoted by -1.0 score.

Click on the column header to sort the table. Hold down the Shift key and select a second column to sort by multiple criteria.