Simple python lib to tokenize texts into sentences and sentences to words. Small, fast and robust. Comes with ukrainian flavour
- Free software: MIT license
- Documentation: https://tokenize_uk.readthedocs.org.
- Tokenize given text into sentences
- Tokenize given sentence into words
- Works well with accented characters (like stresses) and apostrophes
- Suitable also for other languages