Let’s say you’re working on a project that needs to clean up web pages, but you don’t know websites that need to be pre-cleaned beforehand, instead you need to do a google search and then go to google search results for multiple websites. In this case, you need a google search result for your different queries.
- One way to achieve this is to use query and nice soup, which was discussed here at section Implementing Web Scraping in Python with BeautifulSoup .
- Instead of putting so much effort into a trivial task, Google the package was done. This is almost one linear solution to find links to all Google search results directly.
- Using the Google Python package we can get the Google search result from a Python script ... We can get the link from the first n search results.
Install
The Google package has one dependency on Beautifulsoup, which must be installed first.
pip install beautifulsoup4
Then install the google package
pip install google
Required function and its parameters
search (query, tld = ’com’, lang =’ en’, num = 10, start = 0, stop = None, pause = 2.0)
- query: the query string we want to find.
- tld: tld means top-level domain, which means we want to search for our result on google.com , google.in or some other domain.
- lang: lang means language.
- num: number of results, which we want.
- start: first result to fetch.
- stop: last result to fetch. Use None to keep searching forever.
- pause: break between HTTP requests. A mistake too short can cause Google to block your IP. Keeping a significant amount of time will make your program slow, but it is the safest and best option.
- Return: A generator (iterator) that returns the URLs it finds. If stop is None, the iterator will loop forever.
Python codes on how to do a Google search using a Python script
Example 1: google_search.py
|
Output:
Lets you do a manual Google search and check our result
Example 2: google_search.py
|
Output:
Lets do a manual Google search and check our result
Link: Google Python package
This article courtesy of Pratik Chhajer . If you are as Python.Engineering and would like to contribute, you can also write an article using contribute.python.engineering or by posting the article [email protected] ... See my article appearing on the Python.Engineering homepage and help other geeks.
Please write in comments if you find anything wrong or if you’d like to share more information on the topic discussed above.