Change language

Python NLTK | tokenize.regexp ()

| |

Using the NLTK tokenize.regexp () module, we can extract tokens from a string using a regex with RegexpTokenizer () .

Syntax: tokenize.RegexpTokenizer ()
Return: Return array of tokens using regular expression

Example # 1:
In this example we use RegexpTokenizer () to retrieve the stream tokens using regular expressions.

# import RegexpTokenizer () method from nltk

from nltk.tokenize import RegexpTokenizer

 
# Create a reference variable for the RegexpT class okenizer

tk = RegexpTokenizer ( ’s +’ , gaps = True )

  
# Create input string

gfg = "I love Python"

 
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print ( geek)

Output:

[’I’, ’love’, ’Python’]

P Example # 2:

# import RegexpTokenizer () method from nltk

from nltk.tokenize import RegexpTokenizer

 
# Create a reference variable for the RegexpTokenizer class

tk = RegexpTokenizer ( ’s +’ , gaps = True )

  
# Create input line

gfg = "Geeks for Geeks"

  
# Use the tokenization method

geek = tk.tokenize (gfg)

 

print (geek)

Output:

[’Geeks’, ’for’, ’Geeks’]

Shop

Learn programming in R: courses

$

Best Python online courses for 2022

$

Best laptop for Fortnite

$

Best laptop for Excel

$

Best laptop for Solidworks

$

Best laptop for Roblox

$

Best computer for crypto mining

$

Best laptop for Sims 4

$

Latest questions

NUMPYNUMPY

psycopg2: insert multiple rows with one query

12 answers

NUMPYNUMPY

How to convert Nonetype to int or string?

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Javascript Error: IPython is not defined in JupyterLab

12 answers

News


Wiki

Python OpenCV | cv2.putText () method

numpy.arctan2 () in Python

Python | os.path.realpath () method

Python OpenCV | cv2.circle () method

Python OpenCV cv2.cvtColor () method

Python - Move item to the end of the list

time.perf_counter () function in Python

Check if one list is a subset of another in Python

Python os.path.join () method