Change language

NLP | Backoff Tagging for combining taggers

| | | |

What is part-of-speech (POS) tagging?
This is the process of converting sentences to forms — a list of words, a list of tuples (where each tuple is of the form (word, tag)). The tag in the case of a word is a part of speech tag and indicates whether the word is a noun, adjective, verb, etc.

What is Backoff Tagging?
This is one of the most important functions of SequentialBackoffTagger, as it allows taggers to be combined together. The advantage of this is that if the tagger is unaware of the tagging of a word, it can pass this tagging task to the next return tag. If he cannot do this, he can pass the word
the next return tag, and so on until it remains to check the return tags.

Code # 1: Executing tags

# Loading libraries

from nltk.tag import SequentialBackoffTagger

from nltk.tag import DefaultTagger 

from nltk.tag import UnigramTagger 

 

from nltk.corpus import treebank

 
# initialize learning and testing the suite

train_data = treebank.tagged_sents () [: 3000 ]

test_data = treebank.tagged_sents () [ 3000 :]

 
# Tag definition

tag1 = DefaultTagger ( ’NN’ )

  
# Tag

tag2 = Unig ramTagger (train_data, backoff = tag1)

 
# Rating
tag2.evaluate (test_data)

Output:

 0.8752428232246924 

How does it work?
The SequentialBackoffTagger class can take a backoff keyword argument, whose value is another instance of SequentialBackoffTagger. In the above code, the unigram part-of-speech tagger is rolled back with the Default tag and trained on the treebank.tagged_sents () dataset .

Code # 2 : Preparing an internal list of rollback tags

from nltk.tag import SequentialBackoffTagger

 

print (tag1._taggers = = [tag1])

 

print ( "" , tag2. _taggers = = [tag2, tag1])

Output:

 True True 

How does it work?

  • The SequentialBackoffTagger class is initialized, creating an internal list of backoff tags with the first element being itself.
  • An internal list of backoff tags is added if the backoff tag is specified.
  • The SequentialBackoffTagger class uses _taggers list — an internal list of rollback tags when the tag () method is called.
  • By calling choose_tag () on each one, it loops through its own tag list.
  • Stops and returns a tag when a tag is found.
  • The tag will be returned if the main tagger can tag the word.
  • Otherwise, it returns None and tries the next tag, and so on, until the tag is found, or None is returned .

Code # 3: Save and load trained brine tagger.

# Loading Libraries

import pickle

 
# Open file and write

file = open ( ’ tagger.pickle’ , ’wb’ )

pickle.dump (tagger, file )

file . close ()

 
# Reading the file

file = open ( ’tagger.pickle ’ , ’ rb’ )

# Loading

tagger = pickle.load (f)

Output:

 nltk.data.load (’tagger.pickle’) will load the file 

Shop

Learn programming in R: courses

$

Best Python online courses for 2022

$

Best laptop for Fortnite

$

Best laptop for Excel

$

Best laptop for Solidworks

$

Best laptop for Roblox

$

Best computer for crypto mining

$

Best laptop for Sims 4

$

Latest questions

NUMPYNUMPY

Common xlabel/ylabel for matplotlib subplots

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

12 answers

NUMPYNUMPY

Flake8: Ignore specific warning for entire file

12 answers

NUMPYNUMPY

glob exclude pattern

12 answers

NUMPYNUMPY

How to avoid HTTP error 429 (Too Many Requests) python

12 answers

NUMPYNUMPY

Python CSV error: line contains NULL byte

12 answers

NUMPYNUMPY

csv.Error: iterator should return strings, not bytes

12 answers

News


Wiki

Python | How to copy data from one Excel sheet to another

Common xlabel/ylabel for matplotlib subplots

Check if one list is a subset of another in Python

sin

How to specify multiple return types using type-hints

exp

Printing words vertically in Python

exp

Python Extract words from a given string

Cyclic redundancy check in Python

Finding mean, median, mode in Python without libraries

cos

Python add suffix / add prefix to strings in a list

Why do I get "Pickle - EOFError: Ran out of input" reading an empty file?

Python - Move item to the end of the list

Python - Print list vertically