Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote298 views1 answer
What is kernel_initializer
Python
0 vote291 views1 answer
Interval time specification
Python
2 votes291 views6 answers
Python virtual env
Python
1 vote299 views4 answers
Get python script path
Python
0 vote293 views1 answer
Numpy annotate with three arrows
Python
2 votes272 views1 answer
Renaming row value in pandas
Python
2 votes310 views3 answers
Request session python
Python
0 vote333 views1 answer
Python remove articles from string regex
Python
0 vote308 views1 answer
Install older version of python
Python
4 votes294 views2 answers
Python replace by dictionary
Python