Python NLTK | nltk.tokenize.SpaceTokenizer ()

Python Methods and Functions

With the nltk.tokenize.SpaceTokenizer() method we can extract tokens from the word chain based on the space between them using tokenize.SpaceTokenizer () .

Syntax: tokenize.SpaceTokenizer ()
Return: Return the tokens of words.

Example # 1:
In this example we can see that with tokenize.SpaceTokenizer () we can extract tokens from the stream into words containing spaces between them.

# import the SpaceTokenizer () method from nltk

from nltk.tokenize import SpaceTokenizer

  
# Create a link spaceTokenizer

tk = SpaceTokenizer ()

 
# Create input line

gfg = "Geeksfor Geeks ... $$ & amp; * is for geeks "

  
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print (geek)

Output:

[`Geeksfor`, `Geeks ..`, `. $$ & amp; *`, `is`, `for`, `geeks`]

Example # 2:

# import the SpaceTokenizer () method from nltk

from nltk.tokenize import SpaceTokenizer

 
# Create a reference variable for the SpaceTokenizer class

tk = SpaceTokenizer ()

 
# Create input line

gfg = "The price of burger in BurgerKing is Rs. 36. "

  
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print (geek)

Output:

[`The`, `price`, `of`, `burger`, `in`, `BurgerKing`, `is`, `Rs.36.`]





Get Solution for free from DataCamp guru