Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

25 votes297 views3 answers
Startapp django
Python Django
72 votes236 views6 answers
Python loop array
Python
0 vote220 views1 answer
Del en python
Python
0 vote210 views1 answer
Arabic in python
Python
-1 vote242 views2 answers
Django connection cursor
Python Django
0 vote240 views2 answers
5.2.5: Counting 10 to 100 by ...
Python
18 votes173 views9 answers
Dictionary in python
Python
0 vote214 views2 answers
List comprehension with square numbers python
Python
0 vote215 views1 answer
Python script to convert dicom to niftii
Python
2 votes220 views1 answer
Armstrong number python
Python