Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

39 votes318 views6 answers
Pandas count nans in column
Python
0 vote368 views1 answer
Python get weather temperature
Python
68 votes329 views6 answers
Python dict sort by value
Python
1 vote344 views1 answer
All subarrays of an array python
Python
4 votes321 views1 answer
End python in command
Python
0 vote354 views1 answer
Python amino acid dictionary
Python
0 vote320 views1 answer
Search bar using python
Python
2 votes325 views2 answers
Forrest Gump
Python
2 votes332 views3 answers
Local variable
Python
0 vote314 views1 answer
Input character in python like getchar in c
Python