Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote277 views1 answer
Time track python
Python
0 vote255 views1 answer
Class inside class python
Python
1 vote259 views1 answer
Create iterator object python
Python
0 vote309 views1 answer
Production mode flask
Python Flask
2 votes247 views2 answers
Python infinite l00p
Python
2 votes266 views1 answer
Read database pandas
Python
0 vote249 views1 answer
Screen.blit() arguments
Python
0 vote278 views2 answers
How to download excel file with password from online python
Python
1 vote266 views1 answer
Split string form url last slash
Python
1 vote252 views1 answer
How to wirte a function in python
Python