WMT24: General MT Task
The main shared task organized within the 9th Conference on Machine Translation (WMT24). The task evaluates general machine translation capabilities for various domains, genres, and modalities. Submitted systems will be scored and ranked by human judgement.
The official website: www2.statmt.org/wmt24/translation-task.html
Leaderboard of WMT24: General MT Task
wmttest2024.cs-uk test set (cs-uk)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #50 | 32.9 | 60.8 | July 2, 2024, 9:11 a.m. |
| 2 | Anonymous submission #384 | 32.4 | 60.4 | July 4, 2024, 5:07 p.m. |
| 3 | Anonymous submission #33 | 32.4 | 59.4 | July 1, 2024, 8:22 p.m. |
| 4 | Anonymous submission #271 | 32.4 | 59.4 | July 4, 2024, 1:31 p.m. |
| 5 | Anonymous submission #225 | 32.4 | 59.4 | July 4, 2024, 8:17 a.m. |
| 6 | Anonymous submission #211 | 31.5 | 59.2 | July 4, 2024, 2:01 a.m. |
| 7 | Anonymous submission #129 | 30.1 | 59.1 | July 3, 2024, 1:17 p.m. |
| 8 | Anonymous submission #25 | 30.5 | 59.1 | July 1, 2024, 4:23 p.m. |
| 9 | Anonymous submission #179 | 30.7 | 59.0 | July 3, 2024, 4:44 p.m. |
| 10 | Anonymous submission #98 | 30.8 | 58.8 | July 3, 2024, 12:40 a.m. |
wmttest2024.en-cs test set (en-cs)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #76 | 31.3 | 58.4 | July 2, 2024, 5:50 p.m. |
| 2 | Anonymous submission #77 | 31.3 | 58.4 | July 2, 2024, 5:59 p.m. |
| 3 | Anonymous submission #117 | 31.3 | 58.4 | July 3, 2024, 11:23 a.m. |
| 4 | Anonymous submission #34 | 31.6 | 58.4 | July 1, 2024, 8:47 p.m. |
| 5 | Anonymous submission #32 | 31.0 | 57.7 | July 1, 2024, 8:15 p.m. |
| 6 | Anonymous submission #31 | 31.2 | 57.5 | July 1, 2024, 8:11 p.m. |
| 7 | Anonymous submission #67 | 31.4 | 57.1 | July 2, 2024, 2:07 p.m. |
| 8 | Anonymous submission #68 | 30.5 | 56.5 | July 2, 2024, 2:17 p.m. |
| 9 | Anonymous submission #397 | 29.3 | 56.4 | July 4, 2024, 5:11 p.m. |
| 10 | Anonymous submission #249 | 29.0 | 56.3 | July 4, 2024, 11:31 a.m. |
wmttest2024.en-de test set (en-de)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #43 | 34.6 | 61.7 | July 2, 2024, 8:03 a.m. |
| 2 | Anonymous submission #130 | 33.1 | 61.5 | July 3, 2024, 1:45 p.m. |
| 3 | Anonymous submission #42 | 32.6 | 60.9 | July 2, 2024, 7:59 a.m. |
| 4 | Anonymous submission #44 | 33.6 | 60.8 | July 2, 2024, 9:06 a.m. |
| 5 | Anonymous submission #23 | 32.6 | 60.3 | July 1, 2024, 4:20 p.m. |
| 6 | Anonymous submission #216 | 32.5 | 59.9 | July 4, 2024, 6:41 a.m. |
| 7 | Anonymous submission #167 | 32.0 | 59.6 | July 3, 2024, 4:40 p.m. |
| 8 | Anonymous submission #399 | 31.9 | 59.6 | July 4, 2024, 5:15 p.m. |
| 9 | Anonymous submission #54 | 32.3 | 59.5 | July 2, 2024, 9:41 a.m. |
| 10 | Anonymous submission #456 | 31.2 | 59.5 | July 5, 2024, 3:26 p.m. |
wmttest2024.en-es test set (en-es)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #268 | 47.1 | 69.6 | July 4, 2024, 1:20 p.m. |
| 2 | Anonymous submission #160 | 47.1 | 69.6 | July 3, 2024, 4:33 p.m. |
| 3 | Anonymous submission #159 | 47.1 | 69.6 | July 3, 2024, 4:31 p.m. |
| 4 | Anonymous submission #235 | 47.1 | 69.5 | July 4, 2024, 9:59 a.m. |
| 5 | Anonymous submission #415 | 46.8 | 69.5 | July 4, 2024, 5:27 p.m. |
| 6 | Anonymous submission #335 | 46.8 | 69.5 | July 4, 2024, 4:17 p.m. |
| 7 | Anonymous submission #36 | 47.2 | 69.4 | July 1, 2024, 9:03 p.m. |
| 8 | Anonymous submission #412 | 46.8 | 69.3 | July 4, 2024, 5:23 p.m. |
| 9 | Anonymous submission #71 | 46.5 | 69.3 | July 2, 2024, 5:15 p.m. |
| 10 | Anonymous submission #35 | 46.3 | 69.2 | July 1, 2024, 9:02 p.m. |
wmttest2024.en-hi test set (en-hi)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #38 | 27.4 | 53.3 | July 1, 2024, 9:27 p.m. |
| 2 | Anonymous submission #213 | 27.1 | 52.8 | July 4, 2024, 6:20 a.m. |
| 3 | Anonymous submission #109 | 26.4 | 52.4 | July 3, 2024, 9:36 a.m. |
| 4 | Anonymous submission #65 | 25.4 | 51.6 | July 2, 2024, 1:24 p.m. |
| 5 | Anonymous submission #176 | 24.3 | 51.2 | July 3, 2024, 4:42 p.m. |
| 6 | Anonymous submission #27 | 24.4 | 51.1 | July 1, 2024, 4:24 p.m. |
| 7 | Anonymous submission #388 | 24.0 | 51.1 | July 4, 2024, 5:08 p.m. |
| 8 | Anonymous submission #238 | 24.5 | 50.5 | July 4, 2024, 10:20 a.m. |
| 9 | Anonymous submission #128 | 23.3 | 50.5 | July 3, 2024, 1:17 p.m. |
| 10 | Anonymous submission #37 | 24.7 | 50.4 | July 1, 2024, 9:26 p.m. |
wmttest2024.en-is test set (en-is)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #462 | 23.5 | 49.5 | July 5, 2024, 5:03 p.m. |
| 2 | Anonymous submission #390 | 23.4 | 48.4 | July 4, 2024, 5:09 p.m. |
| 3 | Anonymous submission #66 | 23.0 | 48.3 | July 2, 2024, 1:24 p.m. |
| 4 | Anonymous submission #175 | 22.4 | 48.2 | July 3, 2024, 4:42 p.m. |
| 5 | Anonymous submission #115 | 21.5 | 47.8 | July 3, 2024, 11:14 a.m. |
| 6 | Anonymous submission #336 | 21.6 | 47.7 | July 4, 2024, 4:25 p.m. |
| 7 | Anonymous submission #248 | 21.5 | 47.7 | July 4, 2024, 11:25 a.m. |
| 8 | Anonymous submission #21 | 21.5 | 47.6 | July 1, 2024, 4:14 p.m. |
| 9 | Anonymous submission #321 | 21.6 | 47.6 | July 4, 2024, 4 p.m. |
| 10 | Anonymous submission #52 | 21.6 | 47.6 | July 2, 2024, 9:29 a.m. |
wmttest2024.en-ja test set (en-ja)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #475 | 42.6 | 37.1 | July 5, 2024, 11:48 p.m. |
| 2 | Anonymous submission #47 | 41.7 | 36.0 | July 2, 2024, 9:10 a.m. |
| 3 | Anonymous submission #474 | 39.3 | 35.7 | July 5, 2024, 11:41 p.m. |
| 4 | Anonymous submission #420 | 40.2 | 35.6 | July 4, 2024, 5:29 p.m. |
| 5 | Anonymous submission #465 | 41.3 | 35.6 | July 5, 2024, 6:12 p.m. |
| 6 | Anonymous submission #240 | 39.2 | 35.1 | July 4, 2024, 10:24 a.m. |
| 7 | Anonymous submission #467 | 40.0 | 35.0 | July 5, 2024, 8:58 p.m. |
| 8 | Anonymous submission #173 | 40.5 | 34.9 | July 3, 2024, 4:42 p.m. |
| 9 | Anonymous submission #231 | 39.7 | 34.9 | July 4, 2024, 9:54 a.m. |
| 10 | Anonymous submission #447 | 39.5 | 34.8 | July 5, 2024, 11:57 a.m. |
wmttest2024.en-ru test set (en-ru)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #459 | 25.4 | 53.4 | July 5, 2024, 3:38 p.m. |
| 2 | Anonymous submission #53 | 24.3 | 52.9 | July 2, 2024, 9:39 a.m. |
| 3 | Anonymous submission #221 | 24.3 | 52.9 | July 4, 2024, 7:40 a.m. |
| 4 | Anonymous submission #73 | 24.2 | 52.8 | July 2, 2024, 5:27 p.m. |
| 5 | Anonymous submission #55 | 23.8 | 52.5 | July 2, 2024, 9:42 a.m. |
| 6 | Anonymous submission #63 | 24.0 | 52.4 | July 2, 2024, 1:23 p.m. |
| 7 | Anonymous submission #296 | 23.3 | 52.1 | July 4, 2024, 3:26 p.m. |
| 8 | Anonymous submission #392 | 23.4 | 51.7 | July 4, 2024, 5:09 p.m. |
| 9 | Anonymous submission #440 | 21.8 | 51.7 | July 4, 2024, 5:47 p.m. |
| 10 | Anonymous submission #172 | 23.0 | 51.5 | July 3, 2024, 4:41 p.m. |
wmttest2024.en-uk test set (en-uk)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #155 | 32.9 | 58.6 | July 3, 2024, 4 p.m. |
| 2 | Anonymous submission #214 | 32.9 | 58.4 | July 4, 2024, 6:37 a.m. |
| 3 | Anonymous submission #62 | 32.9 | 58.4 | July 2, 2024, 10:13 a.m. |
| 4 | Anonymous submission #157 | 32.9 | 58.4 | July 3, 2024, 4:16 p.m. |
| 5 | Anonymous submission #61 | 33.5 | 58.0 | July 2, 2024, 10:11 a.m. |
| 6 | Anonymous submission #457 | 30.9 | 57.3 | July 5, 2024, 3:37 p.m. |
| 7 | Anonymous submission #458 | 30.9 | 57.3 | July 5, 2024, 3:38 p.m. |
| 8 | Anonymous submission #46 | 31.3 | 56.4 | July 2, 2024, 9:09 a.m. |
| 9 | Anonymous submission #418 | 30.4 | 56.2 | July 4, 2024, 5:28 p.m. |
| 10 | Anonymous submission #170 | 29.9 | 55.9 | July 3, 2024, 4:41 p.m. |
wmttest2024.en-zh test set (en-zh)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #222 | 46.8 | 43.1 | July 4, 2024, 7:58 a.m. |
| 2 | Anonymous submission #293 | 46.8 | 43.1 | July 4, 2024, 3:04 p.m. |
| 3 | Anonymous submission #212 | 46.7 | 42.8 | July 4, 2024, 3:46 a.m. |
| 4 | Anonymous submission #294 | 46.5 | 42.4 | July 4, 2024, 3:13 p.m. |
| 5 | Anonymous submission #241 | 45.7 | 42.4 | July 4, 2024, 10:25 a.m. |
| 6 | Anonymous submission #64 | 44.4 | 40.5 | July 2, 2024, 1:23 p.m. |
| 7 | Anonymous submission #243 | 44.3 | 40.1 | July 4, 2024, 10:32 a.m. |
| 8 | Anonymous submission #208 | 43.6 | 40.0 | July 4, 2024, 1:29 a.m. |
| 9 | Anonymous submission #57 | 43.1 | 39.7 | July 2, 2024, 9:49 a.m. |
| 10 | Anonymous submission #19 | 42.5 | 39.2 | July 1, 2024, 4:12 p.m. |
wmttest2024.ja-zh test set (ja-zh)
| # | Name | BLEU | chrF | Date |
|---|---|---|---|---|
| 1 | Anonymous submission #242 | 32.9 | 29.4 | July 4, 2024, 10:26 a.m. |
| 2 | Anonymous submission #45 | 32.3 | 29.1 | July 2, 2024, 9:09 a.m. |
| 3 | Anonymous submission #233 | 32.3 | 29.0 | July 4, 2024, 9:56 a.m. |
| 4 | Anonymous submission #207 | 32.0 | 28.7 | July 4, 2024, 1:28 a.m. |
| 5 | Anonymous submission #56 | 31.6 | 28.6 | July 2, 2024, 9:45 a.m. |
| 6 | Anonymous submission #398 | 31.2 | 28.0 | July 4, 2024, 5:13 p.m. |
| 7 | Anonymous submission #18 | 30.9 | 27.7 | July 1, 2024, 4:10 p.m. |
| 8 | Anonymous submission #131 | 29.6 | 26.7 | July 3, 2024, 2:04 p.m. |
| 9 | Anonymous submission #9 | 29.6 | 26.7 | July 1, 2024, 12:10 p.m. |
| 10 | Anonymous submission #166 | 29.2 | 26.5 | July 3, 2024, 4:40 p.m. |