Change language

Python NLTK | nltk.WhitespaceTokenizer

Using the nltk.tokenize.WhitespaceTokenizer() method we can extract tokens from a string of words or sentences without spaces , new and tabs strings with tokenize.WhitespaceTokenizer () .

Syntax: tokenize.WhitespaceTokenizer ()
Return: Return the tokens from a string

Example # 1:
In this example, we see that using tokenize.WhitespaceTokenizer () we can extract tokens from the word stream.

# import WhitespaceTokenizer () method from nltk

from nltk .tokenize import WhitespaceTokenizer

  
# Create a reference variable for the WhitespaceTokenizer class

tk = WhitespaceTokenizer ()

 
# Create input line

gfg = "Python.Engineering is for geeks"

 
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print (geek)

Output:

[’GeeksforGeeks’, ’is’, ’for’, ’geeks’]

Example No. 2:

# import WhitespaceTokenizer () method from nltk

from nltk.tokenize import WhitespaceTokenizer

 
# Create a reference variable for the WhitespaceTokenizer class

tk = WhitespaceTokenizer ()

  
# Create input line

gfg = "The price of burger in BurgerKing is Rs. 36. "

  
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print (geek)

Output:

[’The’, ’price’, ’of’, ’burger’, ’in’, ’BurgerKing’, ’is’, ’Rs.36.’]

Shop

Gifts for programmers

Learn programming in R: courses

$FREE
Gifts for programmers

Best Python online courses for 2022

$FREE
Gifts for programmers

Best laptop for Fortnite

$399+
Gifts for programmers

Best laptop for Excel

$
Gifts for programmers

Best laptop for Solidworks

$399+
Gifts for programmers

Best laptop for Roblox

$399+
Gifts for programmers

Best computer for crypto mining

$499+
Gifts for programmers

Best laptop for Sims 4

$

Latest questions

PythonStackOverflow

Common xlabel/ylabel for matplotlib subplots

1947 answers

PythonStackOverflow

Check if one list is a subset of another in Python

1173 answers

PythonStackOverflow

How to specify multiple return types using type-hints

1002 answers

PythonStackOverflow

Printing words vertically in Python

909 answers

PythonStackOverflow

Python Extract words from a given string

798 answers

PythonStackOverflow

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

606 answers

PythonStackOverflow

Python os.path.join () method

384 answers

PythonStackOverflow

Flake8: Ignore specific warning for entire file

360 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

How to specify multiple return types using type-hints

Printing words vertically in Python

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically