zemberek-nlp

I feel like Pardus project has lost some momentum. Maybe I feel like this because there was a lot of publicity and hype around Pardus project when I was growing up. What interested me most is that Pardus project had its own packaged manager called PiSi, which was then adopted by Solus. I bet Pardus project had their own reasons to adopt the apt system and rely on Debian volunteers for all packaging support. As the OS is largely maintained by a government institution, I'd imagine it doesn't have the large userbase as Debian does. There are also some other cool projects made for Pardus, for example Zemberek, a spell checker for the Turkish language that uses natural language processing for Libreoffice and Openoffice: https://github.com/ahmetaa/zemberek-nlp

These are common problems for languages with complex morphology. Finnish, Japanese, Turkish and Hungarian are closely related in this sense and share a lot of NLP research. While implementing spell checkers for these languages need a bit more effort than just compiling a list of words, it's far from an unsolved problem. AFAIK Zemberek[1] is a Turkish spellchecker that implements two-level finite state morphology. Also, a small terminology fix: Verbs are "conjugated" but names/objects are "declinated". [1]: https://github.com/ahmetaa/zemberek-nlp

Not the same technique but generation of words is possible with Zemberek library as well, see the bottom example in the morphology section: https://github.com/ahmetaa/zemberek-nlp/tree/master/morpholo...

For what its worth, I have a quite fast MPHF implementation that uses an average of 3.2 bits per key. https://github.com/ahmetaa/zemberek-nlp/blob/master/core/src... it is used for language model compression. Also there is a Dart implementation as part of a small library. https://github.com/ahmetaa/poppy

Language guessing is rather hard when few letters are used especially if you use statistical methods. I think after 20 something letters you enter >%95 accuracy zone. In a simple library I wrote ( https://github.com/ahmetaa/zemberek-nlp/tree/master/lang-id Works for 60 languages but no docs yet) , for Turkish and English test results are: For 20 letters TR=95.90 EN=94.96 For 50 Letters TR=99.44 EN=99.53 If 50 letters are used in a Doc, it identifies about 20000 docs per second in a decent desktop.