![Checkout](https://naologiccom.imgix.net/website-update/general/checkout.png?auto=compress&w=64&fm=png)
Start free trial
Take Naologic for a spin today, no credit card needed and no obligations.
Start free trial Question
Tokenization - How do you do tokenization in NLP?
Answer
Tokenization is a technique used in Natural Language Processing (NLP) that uses whitespace in a string to delimit words. Any Python string object or the built-in string class can have its contents split using the split method. Any necessary adjustments can be made to the separator.