Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote239 views1 answer
Import matplotlib sub
Python
0 vote248 views1 answer
Numpy savetext in one line
Python
1 vote241 views2 answers
Python string copy
Python
-2 votes245 views1 answer
Python quiz answer stores
Python
1 vote255 views1 answer
Combining series to a dataframe
Python
0 vote206 views1 answer
Duur wordt voor woorden kennis
Python
1 vote272 views1 answer
Why men are better than woman
Python
0 vote265 views1 answer
Excel write in row
Python
23 votes285 views3 answers
How to print a random part of a list in python
Python
1 vote237 views1 answer
Python npr permutation calculation
Python