Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote288 views1 answer
Onetoone vs foreign key django
Python
2 votes284 views2 answers
Get rid of unnamed column pandas
Python
0 vote290 views1 answer
Django.core.exceptions.ImproperlyConfigured: Application labels aren't unique, duplicates: auth
Python
3 votes271 views1 answer
Pi in python math
Python
13 votes279 views3 answers
Pandas reset index start from 0
Python
0 vote267 views1 answer
Opencv save image rgb
Python
0 vote280 views2 answers
Terneray operator in python
Python
1 vote299 views1 answer
Python remove lines of string
Python
-1 vote283 views2 answers
Django view
Python
12 votes301 views6 answers
How to make a list python
Python