WorldLex: Blog, Twitter and Newspapers Word Frequencies for 66 languages
Worldlex makes available new word frequencies based on Twitter, blog posts, or newspapers for 66 languages. We showed that these frequencies predict lexical decision reaction times similar to the already existing frequencies, or even better than them.
You can now access all the WorlLex databases online !
If you use them, you can cite the following article:
Gimenes, M., & New, B. (2015). Worldlex: Twitter and blog word frequencies for 66 languages. Behavior research methods, 1-10.
These corpora have been collected by Hans Christensen for HCCorpora.
| Language | Year collect | Millions words | Raw Freq | Cleaned Freq |
| Afrikaans | 2011 | 14.3 | Link | Link |
| Albanian | 2012 | 26.2 | Link | Link |
| Amharic | 2013 | 2.4 | Link | |
| Arabic | 2011-12 | 62.7 | Link | Link |
| Armenian | 2011 | 14.6 | Link | Link |
| Azeri | 2012 | 12.7 | Link | Link |
| Bengali | 2012 | 5.8 | Link | Link |
| Bosnian | 2013 | 11.7 | Link | |
| Catalan | 2013 | 19.7 | Link | Link |
| Chinese Simplified | 2011 | 94.1 | Link | |
| Croatian | 2012 | 25.5 | Link | Link |
| Czech | 2011 | 29.4 | Link | Link |
| Danish | 2010-11 | 72 | Link | Link |
| Dutch | 2011-12 | 61.4 | Link | Link |
| English US | 2012 | 104.2 | Link | Link |
| Estonian | 2011 | 29.4 | Link | Link |
| Finnish | 2011 | 26.4 | Link | Link |
| French | 2011-12 | 84.2 | Link | Link |
| Georgian | 2011 | 9.5 | Link | |
| German | 2010-11 | 74.8 | Link | Link |
| Greek | 2011-12 | 57.2 | Link | Link |
| Greenlandic | 2012 | 3.7 | Link | |
| Gujarati | 2011 | 10.2 | Link | Link |
| Hebrew | 2011 | 21.4 | Link | Link |
| Hindi | 2011-12 | 13.7 | Link | Link |
| Hungarian | 2011-13 | 66.9 | Link | Link |
| Icelandic | 2011 | 16.7 | Link | Link |
| Indonesian | 2011-12 | 115.2 | Link | Link |
| Italian | 2011-12 | 79.5 | Link | Link |
| Japanese | 2011 | 40.9 | Link | |
| Kannada | 2011 | 9.1 | Link | |
| Kazakh | 2012 | 5.6 | Link | Link |
| Khmer | 2012 | 6 | Link | |
| Korean | 2011 | 55.7 | Link | |
| Latvian | 2012 | 36.3 | Link | Link |
| Lithuanian | 2011 | 9.9 | Link | Link |
| Macedonian | 2012 | 15.4 | Link | Link |
| Malayalam | 2011 | 4 | Link | Link |
| Malaysian | 2011 | 23.9 | Link | |
| Mongolian | 2012 | 10 | Link | Link |
| Nepali | 2013 | 5.9 | Link | Link |
| Norwegian | 2011 | 44 | Link | Link |
| Persian | 2012 | 8.8 | Link | Link |
| Polish | 2011-13 | 74.8 | Link | Link |
| Portuguese Brazil | 2011 | 50.7 | Link | Link |
| Portuguese Europe | 2011-12 | 68 | Link | Link |
| Punjabi | 2012 | 42.2 | Link | Link |
| Romanian | 2011-13 | 74.6 | Link | Link |
| Russian | 2011-12 | 63.6 | Link | Link |
| Serbian (Latin) | 2013 | 21 | Link | Link |
| Sinhala | 2011 | 10.8 | Link | Link |
| Slovak | 2011 | 22.7 | Link | Link |
| Slovenian | 2012-13 | 35.5 | Link | Link |
| Spanish South America | 2012 | 45.8 | Link | Link |
| Spanish Spain | 2011-12 | 45.6 | Link | Link |
| Swahili | 2012 | 13.4 | Link | Link |
| Swedish | 2011-12 | 74.1 | Link | Link |
| Tagalog | 2012 | 13.9 | Link | |
| Tamil | 2011-12 | 10.8 | Link | |
| Telugu | 2011-12 | 9.9 | Link | Link |
| Turkish | 2011-12 | 63.7 | Link | Link |
| Ukrainian | 2011 | 28.8 | Link | Link |
| Urdu | 2012 | 7.7 | Link | |
| Uzbek | 2012 | 5.1 | Link | |
| Vietnamese | 2012 | 46.1 | Link | Link |
| Welsh | 2013 | 3.8 | Link | Link |