Unsupervised and Very Low Resource MT
There is no machine translation available for most of the ~7000 languages spoken on the planet Earth. In the unsupervised and very low resource translation task, we collaborate with the local communities in providing resources and developing MT systems for local minority languages. Like last year, the task will include translation of Upper Sorbian, a minority Slavic language spoken in Germany. This year, we added a translation of closely related Lower Sorbian and Chuvash, a minority Turcic language spoken in south Russia. This year, the tasks are: Unsupervised Machine Translation: German to Lower Sorbian. Lower Sorbian to German. Very Low Resource Supervised Machine Translation: German to Upper Sorbian. Upper Sorbian to German. Low Resource Supervised Machine Translation: Russian to Chuvash. Chuvash to Russian.
Leaderboard of Unsupervised and Very Low Resource MT
Chuvash to Russian test set (chv-ru)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #66 | 23.4 | 0.476 | July 23, 2021, 2:24 p.m. |
2 | Anonymous submission #55 | 23.4 | 0.476 | July 23, 2021, noon |
3 | Anonymous submission #44 | 22.0 | 0.463 | July 23, 2021, 7:31 a.m. |
4 | Anonymous submission #49 | 21.3 | 0.460 | July 23, 2021, 7:53 a.m. |
German to Lower Sorbian test set (de-dsb)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #72 | 29.9 | 0.599 | July 23, 2021, 4:24 p.m. |
2 | Anonymous submission #67 | 29.9 | 0.599 | July 23, 2021, 2:47 p.m. |
3 | Anonymous submission #34 | 27.5 | 0.579 | July 22, 2021, 9:34 p.m. |
4 | Anonymous submission #33 | 27.4 | 0.578 | July 22, 2021, 9:34 p.m. |
5 | Anonymous submission #10 | 24.1 | 0.542 | July 21, 2021, 2:48 p.m. |
6 | Anonymous submission #11 | 22.7 | 0.537 | July 21, 2021, 2:55 p.m. |
7 | Anonymous submission #25 | 22.4 | 0.528 | July 22, 2021, 3:06 p.m. |
8 | Anonymous submission #60 | 8.0 | 0.321 | July 23, 2021, 1:07 p.m. |
9 | Anonymous submission #12 | 6.8 | 0.409 | July 21, 2021, 3:13 p.m. |
10 | Anonymous submission #24 | 6.8 | 0.409 | July 22, 2021, 2:57 p.m. |
German to Upper Sorbian test set (de-hsb)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #17 | 66.3 | 0.837 | July 21, 2021, 4:53 p.m. |
2 | Anonymous submission #28 | 66.3 | 0.836 | July 22, 2021, 8:50 p.m. |
3 | Anonymous submission #69 | 66.2 | 0.837 | July 23, 2021, 4:07 p.m. |
4 | Anonymous submission #64 | 65.9 | 0.833 | July 23, 2021, 2:12 p.m. |
5 | Anonymous submission #62 | 65.8 | 0.833 | July 23, 2021, 1:24 p.m. |
6 | Anonymous submission #42 | 65.8 | 0.833 | July 23, 2021, 7:20 a.m. |
7 | Anonymous submission #48 | 65.6 | 0.832 | July 23, 2021, 7:47 a.m. |
8 | Anonymous submission #57 | 65.5 | 0.831 | July 23, 2021, 12:52 p.m. |
9 | Anonymous submission #56 | 65.5 | 0.831 | July 23, 2021, 12:51 p.m. |
10 | Anonymous submission #30 | 63.3 | 0.819 | July 22, 2021, 9:30 p.m. |
Lower Sorbian to German test set (dsb-de)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #71 | 33.5 | 0.638 | July 23, 2021, 4:23 p.m. |
2 | Anonymous submission #68 | 33.5 | 0.638 | July 23, 2021, 3:01 p.m. |
3 | Anonymous submission #35 | 33.3 | 0.620 | July 22, 2021, 9:34 p.m. |
4 | Anonymous submission #27 | 33.3 | 0.637 | July 22, 2021, 5:37 p.m. |
5 | Anonymous submission #36 | 33.2 | 0.618 | July 22, 2021, 9:35 p.m. |
6 | Anonymous submission #21 | 32.4 | 0.622 | July 22, 2021, 11:58 a.m. |
7 | Anonymous submission #20 | 31.6 | 0.612 | July 22, 2021, 11:55 a.m. |
8 | Anonymous submission #26 | 18.7 | 0.498 | July 22, 2021, 3:08 p.m. |
9 | Anonymous submission #18 | 18.3 | 0.493 | July 21, 2021, 5:01 p.m. |
10 | Anonymous submission #51 | 5.9 | 0.316 | July 23, 2021, 11 a.m. |
Russian to Chuvash test set (ru-chv)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #54 | 22.1 | 0.513 | July 23, 2021, 11:24 a.m. |
2 | Anonymous submission #53 | 21.8 | 0.512 | July 23, 2021, 11:19 a.m. |
3 | Anonymous submission #43 | 20.9 | 0.501 | July 23, 2021, 7:30 a.m. |
4 | Anonymous submission #45 | 20.7 | 0.502 | July 23, 2021, 7:31 a.m. |
5 | Anonymous submission #50 | 19.4 | 0.506 | July 23, 2021, 8:47 a.m. |
Upper Sorbian to German test set (hsb-de)
# | Name | BLEU | chrF | Date |
---|---|---|---|---|
1 | Anonymous submission #65 | 67.7 | 0.834 | July 23, 2021, 2:20 p.m. |
2 | Anonymous submission #41 | 67.6 | 0.834 | July 23, 2021, 6:26 a.m. |
3 | Anonymous submission #40 | 67.6 | 0.834 | July 23, 2021, 6:24 a.m. |
4 | Anonymous submission #39 | 67.6 | 0.834 | July 23, 2021, 3:24 a.m. |
5 | Anonymous submission #38 | 67.6 | 0.834 | July 23, 2021, 3:16 a.m. |
6 | Anonymous submission #70 | 67.3 | 0.836 | July 23, 2021, 4:08 p.m. |
7 | Anonymous submission #14 | 67.2 | 0.836 | July 21, 2021, 3:41 p.m. |
8 | Anonymous submission #29 | 67.2 | 0.836 | July 22, 2021, 8:54 p.m. |
9 | Anonymous submission #19 | 67.0 | 0.833 | July 22, 2021, 9:02 a.m. |
10 | Anonymous submission #47 | 64.3 | 0.819 | July 23, 2021, 7:33 a.m. |