wzray/tt9
1
0
Fork 0
Code Activity

Latvian (#737)

Browse source
This commit is contained in:
Dimo Karaivanov 2025-02-23 13:31:51 +02:00 • committed by GitHub
parent cad27907c7
commit 52532a6d26
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 1192092 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/dictionaries/lvWordlistReadme.txt Normal file
View file

@ -0,0 +1,21 @@
Latvian wordlist 1 from: Tēzaurs.lv
Source: https://repository.clarin.lv/repository/xmlui/handle/20.500.12574/119
Version: 2025 (Winter Edition)
License: Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0), http://creativecommons.org/licenses/by-sa/4.0/
Latvian wordlist 2 by: Eymen Efe Altun
Source: https://github.com/eymenefealtun/all-words-in-all-languages
Version: edc173a7554731fe644319796a167eae8a5a8eaf (2024-11-28)
Latvian word list 3 by: CC-100;
Version: 2020
Source: https://data.statmt.org/cc-100/
References (PDF links are available in the source URL):
- Unsupervised Cross-lingual Representation Learning at Scale, Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020.
- CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data, Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020.
Remark: Only the 4-letter or longer words that appear at least 2 times were used.
Word frequencies from LatinIME dictionaries:
Source: https://android.googlesource.com/platform/packages/inputmethods/LatinIME
Version: 66093bf509ea92fa31d796326d5f30a8d9582ffe (2023-12-21)
License: https://android.googlesource.com/platform/packages/inputmethods/LatinIME/+/refs/heads/main/NOTICE