Simple multi-threaded download manager in Python

Download Manager — it is essentially a computer program designed to download individual files from the Internet. Here we are going to create a simple download manager using streams in Python. Using multithreading, a file can be loaded as chunks simultaneously from different threads. To accomplish this, we’re going to create a simple command line tool that takes the url of a file and then downloads it.

Prerequisites
Windows machine with Python installed.

Configure
Download the following packages from the command line.

1. Package Click: Click & # 8212 ; is a Python package for building beautiful command line interfaces with a minimum of code. This is the “Command Line Interface Bundle.”

    pip install click 

2. Request Package: in this tool we are going to download a file based on a URL (HTTP- addresses). Requests — it is an HTTP library written in Python that allows you to send HTTP requests. You can add headers, form data, multipart files, and parameters using simple Python dictionaries and access response data in the same way.

    pip install requests 

3. Package threads: to work with threads, we need a threading package.

    pip install threading 

Implementation

( Note: The program has been broken up to make it easier to understand. Make sure you don’t miss any part of the code while the code is running.)

• Create new Python file in editor
• First import the required packages you need to write to

Note: this code will not work online.   # Import required packages import click import requests import threading   # The code below is used for each section of the processed file # for each branch to download content from the specified # storage space def Handler (start, end, url, filename):   # specify the beginning and end of the file headers = { ’Range’ : ’ bytes =% d-% d’ % (start, end)}   # request the specified part and get into variable r = requests.get (url, headers = headers, stream = True )   # open the file and write the content of the html page # to file. with open (filename, "r + b" ) as fp:   fp.seek (start) var = fp.tell ( ) fp.write (r.content)

Now we’re going to implement the actual functionality in the download_file function.

• First the step is to style the function with click.command () so that we can add command line arguments. We can also provide options for the appropriate commands.
• For our implementation, after entering the –help command, we are going to display the options that can be used. There are two options in our program that you can use. One of them — "Number_of_threads" and the other — "Name". By default, "number_of_threads" is set to 4. To change it, we can specify when the program starts.
• The "name" option is given so that we can give a custom name to the file to be loaded. Function arguments can be specified using click.argument ().
• For our program, we need to specify the URL of the file we want to download.

# Note: this code will not work in the online IDE   @ click . command ( help = "It downloads the specified file with specified name" ) @click . option ( ’ —number_of_threads’ , default = 4 , help = "No of Threads" ) @ click . option ( ’--name’ , type = click.Path (), help = "Name of the file with extension" ) @ click . argument ( ’url_of_file’ , type = click.Path ()) @ click . pass_context def download_file (ctx, url_of_file, name, number_of_threads):

The following code is part of the "download_file" function.

< li> In this function, we first check the "name". If no name is provided, use the name from the URL.
• The next step is to connect to the URL and get the content and size of the content.

r = requests.head (url_of_file) if name: file_name = name else : file_name = url_of_file.split ( ’/’ ) [ - 1 ] try : file_size = int (r.headers [ ’ content-length’ ]) except : print "Invalid URL" return

Create file with content size

part = int (file_size) / number_of_threads fp = open (file_name, "wb" ) fp.write ( ’’ * file_size) fp.close ()

Now we create Threads and pass the Handler function, which has the main functionality:

for i in range (number_of_threads): start = part * i   end = start + part    # create a theme with start and end locations t = threading.Thread (target = Handler,   kwargs = { ’start ’ : start, ’ end’ : end, ’url’ : url_of_file, ’ filename’ : file_name})   t.setDaemon ( True ) t.start ()

Finally join the streams and call function "download_file" from main

main_thread = threading.current_thread () for t in threading. enumerate (): if t is main_thread:   continue t.join () print ’% s downloaded’ % file_name   if __ name__ = = ’__main__’ : download_file (obj = {})

We are done with some of the coding and are now following the commands shown below, to run the .py file.

 “python filename.py” –-help 

This command shows the "Usage" of the click command tool and the options the tool can accept. 
Below is an example of a command where we are trying to load a jpg image file from a URL and also given a name and number_of_threads.

Finally, we have successfully done this, and this is one way to create a simple multi-threaded download manager in Python.

This article contributed by Rahul Boyanapalli . If you are as Python.Engineering and would like to contribute, you can also write an article using contribute.python.engineering or by posting an article contribute @ python.engineering. See my article appearing on the Python.Engineering homepage and help other geeks.

Please post comments if you find anything wrong or if you would like to share more information on the topic discussed above.