How to recursively delete a directory and all its contents (files + subdirectories) in PHP?

| | | | | | | | | |
PHP function to delete all files:The following code first passes the path to the directory to be deleted. It checks if the file or directory to be deleted exists, in fact / exists or not. If it exists, it will open the file and check if there is something in that file or not. If not, then remove the directory using the rmdir directory. But if any other files are present in the directory, then it will delete the files using the unlink function except. and .. files, which mean system files. After removing all content, just use the rmdir function to completely remove the directory.Example : < ? php
// Variable to store the directory name
// to be removed $folder = ’temporary_files’ ;
// Get a list of all filenames // in the folder. $files = glob ( $folder . ’/ *’ );
// Loop through the list of files foreach ( $files as $file ) { // Check for file if ( is_file ( $file )) { // Use the unlink function for // delete the file. unlink (
$file ); } }
?>
  • Before deleting files:
  • After deleting files:
Description :
  • The above example removes all files from the folder named "temp‚".
  • List the files in this directory using the PHP glob function. The glob function basically "finds paths that match a certain pattern". In this case, use the * (asterisk) wildcard to select everything in the temp folder.
  • The glob function returns an array of filenames that are in the specified folder.
  • Loop over this array.
  • Using the is_file function to check if it is a file and not a parent directory or subdirectory.
  • Finally, use the unlink function, which removes the file (if PHP has valid permissions - if not, it will throw an E_WARNING error and return the Boolean value FALSE).
  • Remove all files and subdirectories in a directory.To remove all files and subdirectories in all subdirectories, we can use recursion. Here is an example of a recursive PHP function that deletes every file and folder in the specified directory.Example :
    // Function for deleting all files
    // and directories function deleteAll ( $str ) { // Check files if ( is_file ( $str )) { // If this is a file, then delete // using the unlink function return unlink ( $str ); } // If it’s a directory. elseif ( is_dir ( $str )) { // Get a list of files in this // directory $scan = glob (rtrim ( $str , ’/’ ). ’/ *’ ); // Loop through the list of files foreach ( $scan as $index = > $path ) { // Call a recursive function deleteAll ( $path ); } // Delete the directory itself return @ rmdir ( $str ); } }
    // function call deleteAll ( ’temporary_files’ );
    ?>
    Output:
    • Before deleting the directory:
    • After deleting the directory:the directory is completely deleted.
    The function checks if the variable represents $str is the path to the file, then deletes the file using the unlink function. However, if $str represents a directory, it gets a list of all files in the specified directory before deleting each one. Finally, it removes the subdirectory itself using the PHP rmdir function.

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    How can I make a time delay in Python?

    I would like to know how to put a time delay in a Python script.

    Answer #1:

    import time
    time.sleep(5)   # Delays for 5 seconds. You can also use a float value.
    

    Here is another example where something is run approximately once a minute:

    import time
    while True:
        print("This prints once a minute.")
        time.sleep(60) # Delay for 1 minute (60 seconds).
    

    Answer #2:

    You can use the sleep() function in the time module. It can take a float argument for sub-second resolution.

    from time import sleep
    sleep(0.1) # Time in seconds
    

    Answer #3:

    How can I make a time delay in Python?

    In a single thread I suggest the sleep function:

    >>> from time import sleep
    
    >>> sleep(4)
    

    This function actually suspends the processing of the thread in which it is called by the operating system, allowing other threads and processes to execute while it sleeps.

    Use it for that purpose, or simply to delay a function from executing. For example:

    >>> def party_time():
    ...     print("hooray!")
    ...
    >>> sleep(3); party_time()
    hooray!
    

    "hooray!" is printed 3 seconds after I hit Enter.

    Example using sleep with multiple threads and processes

    Again, sleep suspends your thread - it uses next to zero processing power.

    To demonstrate, create a script like this (I first attempted this in an interactive Python 3.5 shell, but sub-processes can"t find the party_later function for some reason):

    from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor, as_completed
    from time import sleep, time
    
    def party_later(kind="", n=""):
        sleep(3)
        return kind + n + " party time!: " + __name__
    
    def main():
        with ProcessPoolExecutor() as proc_executor:
            with ThreadPoolExecutor() as thread_executor:
                start_time = time()
                proc_future1 = proc_executor.submit(party_later, kind="proc", n="1")
                proc_future2 = proc_executor.submit(party_later, kind="proc", n="2")
                thread_future1 = thread_executor.submit(party_later, kind="thread", n="1")
                thread_future2 = thread_executor.submit(party_later, kind="thread", n="2")
                for f in as_completed([
                  proc_future1, proc_future2, thread_future1, thread_future2,]):
                    print(f.result())
                end_time = time()
        print("total time to execute four 3-sec functions:", end_time - start_time)
    
    if __name__ == "__main__":
        main()
    

    Example output from this script:

    thread1 party time!: __main__
    thread2 party time!: __main__
    proc1 party time!: __mp_main__
    proc2 party time!: __mp_main__
    total time to execute four 3-sec functions: 3.4519670009613037
    

    Multithreading

    You can trigger a function to be called at a later time in a separate thread with the Timer threading object:

    >>> from threading import Timer
    >>> t = Timer(3, party_time, args=None, kwargs=None)
    >>> t.start()
    >>>
    >>> hooray!
    
    >>>
    

    The blank line illustrates that the function printed to my standard output, and I had to hit Enter to ensure I was on a prompt.

    The upside of this method is that while the Timer thread was waiting, I was able to do other things, in this case, hitting Enter one time - before the function executed (see the first empty prompt).

    There isn"t a respective object in the multiprocessing library. You can create one, but it probably doesn"t exist for a reason. A sub-thread makes a lot more sense for a simple timer than a whole new subprocess.

    Answer #4:

    Delays can be also implemented by using the following methods.

    The first method:

    import time
    time.sleep(5) # Delay for 5 seconds.
    

    The second method to delay would be using the implicit wait method:

     driver.implicitly_wait(5)
    

    The third method is more useful when you have to wait until a particular action is completed or until an element is found:

    self.wait.until(EC.presence_of_element_located((By.ID, "UserName"))
    

    How to delete a file or folder in Python?

    How do I delete a file or folder in Python?

    Answer #1:


    Path objects from the Python 3.4+ pathlib module also expose these instance methods:

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    Answer #2:


    Path objects from the Python 3.4+ pathlib module also expose these instance methods:

    Answer #3:

    Python syntax to delete a file

    import os
    os.remove("/tmp/<file_name>.txt")
    

    Or

    import os
    os.unlink("/tmp/<file_name>.txt")
    

    Or

    pathlib Library for Python version >= 3.4

    file_to_rem = pathlib.Path("/tmp/<file_name>.txt")
    file_to_rem.unlink()
    

    Path.unlink(missing_ok=False)

    Unlink method used to remove the file or the symbolik link.

    If missing_ok is false (the default), FileNotFoundError is raised if the path does not exist.
    If missing_ok is true, FileNotFoundError exceptions will be ignored (same behavior as the POSIX rm -f command).
    Changed in version 3.8: The missing_ok parameter was added.

    Best practice

    1. First, check whether the file or folder exists or not then only delete that file. This can be achieved in two ways :
      a. os.path.isfile("/path/to/file")
      b. Use exception handling.

    EXAMPLE for os.path.isfile

    #!/usr/bin/python
    import os
    myfile="/tmp/foo.txt"
    
    ## If file exists, delete it ##
    if os.path.isfile(myfile):
        os.remove(myfile)
    else:    ## Show an error ##
        print("Error: %s file not found" % myfile)
    

    Exception Handling

    #!/usr/bin/python
    import os
    
    ## Get input ##
    myfile= raw_input("Enter file name to delete: ")
    
    ## Try to delete the file ##
    try:
        os.remove(myfile)
    except OSError as e:  ## if failed, report it back to the user ##
        print ("Error: %s - %s." % (e.filename, e.strerror))
    

    RESPECTIVE OUTPUT

    Enter file name to delete : demo.txt
    Error: demo.txt - No such file or directory.
    
    Enter file name to delete : rrr.txt
    Error: rrr.txt - Operation not permitted.
    
    Enter file name to delete : foo.txt
    

    Python syntax to delete a folder

    shutil.rmtree()
    

    Example for shutil.rmtree()

    #!/usr/bin/python
    import os
    import sys
    import shutil
    
    # Get directory name
    mydir= raw_input("Enter directory name: ")
    
    ## Try to remove tree; if failed show an error using try...except on screen
    try:
        shutil.rmtree(mydir)
    except OSError as e:
        print ("Error: %s - %s." % (e.filename, e.strerror))
    

    Answer #4:

    Here is a robust function that uses both os.remove and shutil.rmtree:

    def remove(path):
        """ param <path> could either be relative or absolute. """
        if os.path.isfile(path) or os.path.islink(path):
            os.remove(path)  # remove the file
        elif os.path.isdir(path):
            shutil.rmtree(path)  # remove dir and all contains
        else:
            raise ValueError("file {} is not a file or dir.".format(path))
    

    Is there a simple way to delete a list element by value?

    I want to remove a value from a list if it exists in the list (which it may not).

    a = [1, 2, 3, 4]
    b = a.index(6)
    
    del a[b]
    print(a)
    

    The above case (in which it does not exist) shows the following error:

    Traceback (most recent call last):
      File "D:zjm_codea.py", line 6, in <module>
        b = a.index(6)
    ValueError: list.index(x): x not in list
    

    So I have to do this:

    a = [1, 2, 3, 4]
    
    try:
        b = a.index(6)
        del a[b]
    except:
        pass
    
    print(a)
    

    But is there not a simpler way to do this?

    Answer #1:

    To remove an element"s first occurrence in a list, simply use list.remove:

    >>> a = ["a", "b", "c", "d"]
    >>> a.remove("b")
    >>> print(a)
    ["a", "c", "d"]
    

    Mind that it does not remove all occurrences of your element. Use a list comprehension for that.

    >>> a = [10, 20, 30, 40, 20, 30, 40, 20, 70, 20]
    >>> a = [x for x in a if x != 20]
    >>> print(a)
    [10, 30, 40, 30, 40, 70]
    

    Answer #2:

    Usually Python will throw an Exception if you tell it to do something it can"t so you"ll have to do either:

    if c in a:
        a.remove(c)
    

    or:

    try:
        a.remove(c)
    except ValueError:
        pass
    

    An Exception isn"t necessarily a bad thing as long as it"s one you"re expecting and handle properly.

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    How do I list all files of a directory?

    How can I list all files of a directory in Python and add them to a list?

    Answer #1:

    os.listdir() will get you everything that"s in a directory - files and directories.

    If you want just files, you could either filter this down using os.path:

    from os import listdir
    from os.path import isfile, join
    onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]
    

    or you could use os.walk() which will yield two lists for each directory it visits - splitting into files and dirs for you. If you only want the top directory you can break the first time it yields

    from os import walk
    
    f = []
    for (dirpath, dirnames, filenames) in walk(mypath):
        f.extend(filenames)
        break
    

    or, shorter:

    from os import walk
    
    filenames = next(walk(mypath), (None, None, []))[2]  # [] if no file
    

    Answer #2:

    I prefer using the glob module, as it does pattern matching and expansion.

    import glob
    print(glob.glob("/home/adam/*"))
    

    It does pattern matching intuitively

    import glob
    # All files ending with .txt
    print(glob.glob("/home/adam/*.txt")) 
    # All files ending with .txt with depth of 2 folder
    print(glob.glob("/home/adam/*/*.txt")) 
    

    It will return a list with the queried files:

    ["/home/adam/file1.txt", "/home/adam/file2.txt", .... ]
    

    Answer #3:

    os.listdir() - list in the current directory

    With listdir in os module you get the files and the folders in the current dir

     import os
     arr = os.listdir()
     print(arr)
     
     >>> ["$RECYCLE.BIN", "work.txt", "3ebooks.txt", "documents"]
    

    Looking in a directory

    arr = os.listdir("c:\files")
    

    glob from glob

    with glob you can specify a type of file to list like this

    import glob
    
    txtfiles = []
    for file in glob.glob("*.txt"):
        txtfiles.append(file)
    

    glob in a list comprehension

    mylist = [f for f in glob.glob("*.txt")]
    

    get the full path of only files in the current directory

    import os
    from os import listdir
    from os.path import isfile, join
    
    cwd = os.getcwd()
    onlyfiles = [os.path.join(cwd, f) for f in os.listdir(cwd) if 
    os.path.isfile(os.path.join(cwd, f))]
    print(onlyfiles) 
    
    ["G:\getfilesname\getfilesname.py", "G:\getfilesname\example.txt"]
    

    Getting the full path name with os.path.abspath

    You get the full path in return

     import os
     files_path = [os.path.abspath(x) for x in os.listdir()]
     print(files_path)
     
     ["F:\documentiapplications.txt", "F:\documenticollections.txt"]
    

    Walk: going through sub directories

    os.walk returns the root, the directories list and the files list, that is why I unpacked them in r, d, f in the for loop; it, then, looks for other files and directories in the subfolders of the root and so on until there are no subfolders.

    import os
    
    # Getting the current work directory (cwd)
    thisdir = os.getcwd()
    
    # r=root, d=directories, f = files
    for r, d, f in os.walk(thisdir):
        for file in f:
            if file.endswith(".docx"):
                print(os.path.join(r, file))
    

    os.listdir(): get files in the current directory (Python 2)

    In Python 2, if you want the list of the files in the current directory, you have to give the argument as "." or os.getcwd() in the os.listdir method.

     import os
     arr = os.listdir(".")
     print(arr)
     
     >>> ["$RECYCLE.BIN", "work.txt", "3ebooks.txt", "documents"]
    

    To go up in the directory tree

    # Method 1
    x = os.listdir("..")
    
    # Method 2
    x= os.listdir("/")
    

    Get files: os.listdir() in a particular directory (Python 2 and 3)

     import os
     arr = os.listdir("F:\python")
     print(arr)
     
     >>> ["$RECYCLE.BIN", "work.txt", "3ebooks.txt", "documents"]
    

    Get files of a particular subdirectory with os.listdir()

    import os
    
    x = os.listdir("./content")
    

    os.walk(".") - current directory

     import os
     arr = next(os.walk("."))[2]
     print(arr)
     
     >>> ["5bs_Turismo1.pdf", "5bs_Turismo1.pptx", "esperienza.txt"]
    

    next(os.walk(".")) and os.path.join("dir", "file")

     import os
     arr = []
     for d,r,f in next(os.walk("F:\_python")):
         for file in f:
             arr.append(os.path.join(r,file))
    
     for f in arr:
         print(files)
    
    >>> F:\_python\dict_class.py
    >>> F:\_python\programmi.txt
    

    next(os.walk("F:\") - get the full path - list comprehension

     [os.path.join(r,file) for r,d,f in next(os.walk("F:\_python")) for file in f]
     
     >>> ["F:\_python\dict_class.py", "F:\_python\programmi.txt"]
    

    os.walk - get full path - all files in sub dirs**

    x = [os.path.join(r,file) for r,d,f in os.walk("F:\_python") for file in f]
    print(x)
    
    >>> ["F:\_python\dict.py", "F:\_python\progr.txt", "F:\_python\readl.py"]
    

    os.listdir() - get only txt files

     arr_txt = [x for x in os.listdir() if x.endswith(".txt")]
     print(arr_txt)
     
     >>> ["work.txt", "3ebooks.txt"]
    

    Using glob to get the full path of the files

    If I should need the absolute path of the files:

    from path import path
    from glob import glob
    x = [path(f).abspath() for f in glob("F:\*.txt")]
    for f in x:
        print(f)
    
    >>> F:acquistionline.txt
    >>> F:acquisti_2018.txt
    >>> F:ootstrap_jquery_ecc.txt
    

    Using os.path.isfile to avoid directories in the list

    import os.path
    listOfFiles = [f for f in os.listdir() if os.path.isfile(f)]
    print(listOfFiles)
    
    >>> ["a simple game.py", "data.txt", "decorator.py"]
    

    Using pathlib from Python 3.4

    import pathlib
    
    flist = []
    for p in pathlib.Path(".").iterdir():
        if p.is_file():
            print(p)
            flist.append(p)
    
     >>> error.PNG
     >>> exemaker.bat
     >>> guiprova.mp3
     >>> setup.py
     >>> speak_gui2.py
     >>> thumb.PNG
    

    With list comprehension:

    flist = [p for p in pathlib.Path(".").iterdir() if p.is_file()]
    

    Alternatively, use pathlib.Path() instead of pathlib.Path(".")

    Use glob method in pathlib.Path()

    import pathlib
    
    py = pathlib.Path().glob("*.py")
    for file in py:
        print(file)
    
    >>> stack_overflow_list.py
    >>> stack_overflow_list_tkinter.py
    

    Get all and only files with os.walk

    import os
    x = [i[2] for i in os.walk(".")]
    y=[]
    for t in x:
        for f in t:
            y.append(f)
    print(y)
    
    >>> ["append_to_list.py", "data.txt", "data1.txt", "data2.txt", "data_180617", "os_walk.py", "READ2.py", "read_data.py", "somma_defaltdic.py", "substitute_words.py", "sum_data.py", "data.txt", "data1.txt", "data_180617"]
    

    Get only files with next and walk in a directory

     import os
     x = next(os.walk("F://python"))[2]
     print(x)
     
     >>> ["calculator.bat","calculator.py"]
    

    Get only directories with next and walk in a directory

     import os
     next(os.walk("F://python"))[1] # for the current dir use (".")
     
     >>> ["python3","others"]
    

    Get all the subdir names with walk

    for r,d,f in os.walk("F:\_python"):
        for dirs in d:
            print(dirs)
    
    >>> .vscode
    >>> pyexcel
    >>> pyschool.py
    >>> subtitles
    >>> _metaprogramming
    >>> .ipynb_checkpoints
    

    os.scandir() from Python 3.5 and greater

    import os
    x = [f.name for f in os.scandir() if f.is_file()]
    print(x)
    
    >>> ["calculator.bat","calculator.py"]
    
    # Another example with scandir (a little variation from docs.python.org)
    # This one is more efficient than os.listdir.
    # In this case, it shows the files only in the current directory
    # where the script is executed.
    
    import os
    with os.scandir() as i:
        for entry in i:
            if entry.is_file():
                print(entry.name)
    
    >>> ebookmaker.py
    >>> error.PNG
    >>> exemaker.bat
    >>> guiprova.mp3
    >>> setup.py
    >>> speakgui4.py
    >>> speak_gui2.py
    >>> speak_gui3.py
    >>> thumb.PNG
    

    Examples:

    Ex. 1: How many files are there in the subdirectories?

    In this example, we look for the number of files that are included in all the directory and its subdirectories.

    import os
    
    def count(dir, counter=0):
        "returns number of files in dir and subdirs"
        for pack in os.walk(dir):
            for f in pack[2]:
                counter += 1
        return dir + " : " + str(counter) + "files"
    
    print(count("F:\python"))
    
    >>> "F:\python" : 12057 files"
    

    Ex.2: How to copy all files from a directory to another?

    A script to make order in your computer finding all files of a type (default: pptx) and copying them in a new folder.

    import os
    import shutil
    from path import path
    
    destination = "F:\file_copied"
    # os.makedirs(destination)
    
    def copyfile(dir, filetype="pptx", counter=0):
        "Searches for pptx (or other - pptx is the default) files and copies them"
        for pack in os.walk(dir):
            for f in pack[2]:
                if f.endswith(filetype):
                    fullpath = pack[0] + "\" + f
                    print(fullpath)
                    shutil.copy(fullpath, destination)
                    counter += 1
        if counter > 0:
            print("-" * 30)
            print("	==> Found in: `" + dir + "` : " + str(counter) + " files
    ")
    
    for dir in os.listdir():
        "searches for folders that starts with `_`"
        if dir[0] == "_":
            # copyfile(dir, filetype="pdf")
            copyfile(dir, filetype="txt")
    
    
    >>> _compiti18Compito Contabilità 1conti.txt
    >>> _compiti18Compito Contabilità 1modula4.txt
    >>> _compiti18Compito Contabilità 1moduloa4.txt
    >>> ------------------------
    >>> ==> Found in: `_compiti18` : 3 files
    

    Ex. 3: How to get all the files in a txt file

    In case you want to create a txt file with all the file names:

    import os
    mylist = ""
    with open("filelist.txt", "w", encoding="utf-8") as file:
        for eachfile in os.listdir():
            mylist += eachfile + "
    "
        file.write(mylist)
    

    Example: txt with all the files of an hard drive

    """
    We are going to save a txt file with all the files in your directory.
    We will use the function walk()
    """
    
    import os
    
    # see all the methods of os
    # print(*dir(os), sep=", ")
    listafile = []
    percorso = []
    with open("lista_file.txt", "w", encoding="utf-8") as testo:
        for root, dirs, files in os.walk("D:\"):
            for file in files:
                listafile.append(file)
                percorso.append(root + "\" + file)
                testo.write(file + "
    ")
    listafile.sort()
    print("N. of files", len(listafile))
    with open("lista_file_ordinata.txt", "w", encoding="utf-8") as testo_ordinato:
        for file in listafile:
            testo_ordinato.write(file + "
    ")
    
    with open("percorso.txt", "w", encoding="utf-8") as file_percorso:
        for file in percorso:
            file_percorso.write(file + "
    ")
    
    os.system("lista_file.txt")
    os.system("lista_file_ordinata.txt")
    os.system("percorso.txt")
    

    All the file of C: in one text file

    This is a shorter version of the previous code. Change the folder where to start finding the files if you need to start from another position. This code generate a 50 mb on text file on my computer with something less then 500.000 lines with files with the complete path.

    import os
    
    with open("file.txt", "w", encoding="utf-8") as filewrite:
        for r, d, f in os.walk("C:\"):
            for file in f:
                filewrite.write(f"{r + file}
    ")
    

    How to write a file with all paths in a folder of a type

    With this function you can create a txt file that will have the name of a type of file that you look for (ex. pngfile.txt) with all the full path of all the files of that type. It can be useful sometimes, I think.

    import os
    
    def searchfiles(extension=".ttf", folder="H:\"):
        "Create a txt file with all the file of a type"
        with open(extension[1:] + "file.txt", "w", encoding="utf-8") as filewrite:
            for r, d, f in os.walk(folder):
                for file in f:
                    if file.endswith(extension):
                        filewrite.write(f"{r + file}
    ")
    
    # looking for png file (fonts) in the hard disk H:
    searchfiles(".png", "H:\")
    
    >>> H:4bs_18Dolphins5.png
    >>> H:4bs_18Dolphins6.png
    >>> H:4bs_18Dolphins7.png
    >>> H:5_18marketing htmlassetsimageslogo2.png
    >>> H:7z001.png
    >>> H:7z002.png
    

    (New) Find all files and open them with tkinter GUI

    I just wanted to add in this 2019 a little app to search for all files in a dir and be able to open them by doubleclicking on the name of the file in the list. enter image description here

    import tkinter as tk
    import os
    
    def searchfiles(extension=".txt", folder="H:\"):
        "insert all files in the listbox"
        for r, d, f in os.walk(folder):
            for file in f:
                if file.endswith(extension):
                    lb.insert(0, r + "\" + file)
    
    def open_file():
        os.startfile(lb.get(lb.curselection()[0]))
    
    root = tk.Tk()
    root.geometry("400x400")
    bt = tk.Button(root, text="Search", command=lambda:searchfiles(".png", "H:\"))
    bt.pack()
    lb = tk.Listbox(root)
    lb.pack(fill="both", expand=1)
    lb.bind("<Double-Button>", lambda x: open_file())
    root.mainloop()
    

    Answer #4:

    import os
    os.listdir("somedirectory")
    

    will return a list of all files and directories in "somedirectory".

    Answer #5:

    A one-line solution to get only list of files (no subdirectories):

    filenames = next(os.walk(path))[2]
    

    or absolute pathnames:

    paths = [os.path.join(path, fn) for fn in next(os.walk(path))[2]]
    

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    Finding the index of an item in a list

    Given a list ["foo", "bar", "baz"] and an item in the list "bar", how do I get its index (1) in Python?

    Answer #1:

    >>> ["foo", "bar", "baz"].index("bar")
    1
    

    Reference: Data Structures > More on Lists

    Caveats follow

    Note that while this is perhaps the cleanest way to answer the question as asked, index is a rather weak component of the list API, and I can"t remember the last time I used it in anger. It"s been pointed out to me in the comments that because this answer is heavily referenced, it should be made more complete. Some caveats about list.index follow. It is probably worth initially taking a look at the documentation for it:

    list.index(x[, start[, end]])
    

    Return zero-based index in the list of the first item whose value is equal to x. Raises a ValueError if there is no such item.

    The optional arguments start and end are interpreted as in the slice notation and are used to limit the search to a particular subsequence of the list. The returned index is computed relative to the beginning of the full sequence rather than the start argument.

    Linear time-complexity in list length

    An index call checks every element of the list in order, until it finds a match. If your list is long, and you don"t know roughly where in the list it occurs, this search could become a bottleneck. In that case, you should consider a different data structure. Note that if you know roughly where to find the match, you can give index a hint. For instance, in this snippet, l.index(999_999, 999_990, 1_000_000) is roughly five orders of magnitude faster than straight l.index(999_999), because the former only has to search 10 entries, while the latter searches a million:

    >>> import timeit
    >>> timeit.timeit("l.index(999_999)", setup="l = list(range(0, 1_000_000))", number=1000)
    9.356267921015387
    >>> timeit.timeit("l.index(999_999, 999_990, 1_000_000)", setup="l = list(range(0, 1_000_000))", number=1000)
    0.0004404920036904514
     
    

    Only returns the index of the first match to its argument

    A call to index searches through the list in order until it finds a match, and stops there. If you expect to need indices of more matches, you should use a list comprehension, or generator expression.

    >>> [1, 1].index(1)
    0
    >>> [i for i, e in enumerate([1, 2, 1]) if e == 1]
    [0, 2]
    >>> g = (i for i, e in enumerate([1, 2, 1]) if e == 1)
    >>> next(g)
    0
    >>> next(g)
    2
    

    Most places where I once would have used index, I now use a list comprehension or generator expression because they"re more generalizable. So if you"re considering reaching for index, take a look at these excellent Python features.

    Throws if element not present in list

    A call to index results in a ValueError if the item"s not present.

    >>> [1, 1].index(2)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: 2 is not in list
    

    If the item might not be present in the list, you should either

    1. Check for it first with item in my_list (clean, readable approach), or
    2. Wrap the index call in a try/except block which catches ValueError (probably faster, at least when the list to search is long, and the item is usually present.)

    Answer #2:

    One thing that is really helpful in learning Python is to use the interactive help function:

    >>> help(["foo", "bar", "baz"])
    Help on list object:
    
    class list(object)
     ...
    
     |
     |  index(...)
     |      L.index(value, [start, [stop]]) -> integer -- return first index of value
     |
    

    which will often lead you to the method you are looking for.

    Answer #3:

    The majority of answers explain how to find a single index, but their methods do not return multiple indexes if the item is in the list multiple times. Use enumerate():

    for i, j in enumerate(["foo", "bar", "baz"]):
        if j == "bar":
            print(i)
    

    The index() function only returns the first occurrence, while enumerate() returns all occurrences.

    As a list comprehension:

    [i for i, j in enumerate(["foo", "bar", "baz"]) if j == "bar"]
    

    Here"s also another small solution with itertools.count() (which is pretty much the same approach as enumerate):

    from itertools import izip as zip, count # izip for maximum efficiency
    [i for i, j in zip(count(), ["foo", "bar", "baz"]) if j == "bar"]
    

    This is more efficient for larger lists than using enumerate():

    $ python -m timeit -s "from itertools import izip as zip, count" "[i for i, j in zip(count(), ["foo", "bar", "baz"]*500) if j == "bar"]"
    10000 loops, best of 3: 174 usec per loop
    $ python -m timeit "[i for i, j in enumerate(["foo", "bar", "baz"]*500) if j == "bar"]"
    10000 loops, best of 3: 196 usec per loop
    

    Answer #4:

    To get all indexes:

    indexes = [i for i,x in enumerate(xs) if x == "foo"]
    

    Answer #5:

    index() returns the first index of value!

    | index(...)
    | L.index(value, [start, [stop]]) -> integer -- return first index of value

    def all_indices(value, qlist):
        indices = []
        idx = -1
        while True:
            try:
                idx = qlist.index(value, idx+1)
                indices.append(idx)
            except ValueError:
                break
        return indices
    
    all_indices("foo", ["foo";"bar";"baz";"foo"])
    

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    Meaning of @classmethod and @staticmethod for beginner?

    Question by user1632861

    Could someone explain to me the meaning of @classmethod and @staticmethod in python? I need to know the difference and the meaning.

    As far as I understand, @classmethod tells a class that it"s a method which should be inherited into subclasses, or... something. However, what"s the point of that? Why not just define the class method without adding @classmethod or @staticmethod or any @ definitions?

    tl;dr: when should I use them, why should I use them, and how should I use them?

    Answer #1:

    Though classmethod and staticmethod are quite similar, there"s a slight difference in usage for both entities: classmethod must have a reference to a class object as the first parameter, whereas staticmethod can have no parameters at all.

    Example

    class Date(object):
    
        def __init__(self, day=0, month=0, year=0):
            self.day = day
            self.month = month
            self.year = year
    
        @classmethod
        def from_string(cls, date_as_string):
            day, month, year = map(int, date_as_string.split("-"))
            date1 = cls(day, month, year)
            return date1
    
        @staticmethod
        def is_date_valid(date_as_string):
            day, month, year = map(int, date_as_string.split("-"))
            return day <= 31 and month <= 12 and year <= 3999
    
    date2 = Date.from_string("11-09-2012")
    is_date = Date.is_date_valid("11-09-2012")
    

    Explanation

    Let"s assume an example of a class, dealing with date information (this will be our boilerplate):

    class Date(object):
    
        def __init__(self, day=0, month=0, year=0):
            self.day = day
            self.month = month
            self.year = year
    

    This class obviously could be used to store information about certain dates (without timezone information; let"s assume all dates are presented in UTC).

    Here we have __init__, a typical initializer of Python class instances, which receives arguments as a typical instancemethod, having the first non-optional argument (self) that holds a reference to a newly created instance.

    Class Method

    We have some tasks that can be nicely done using classmethods.

    Let"s assume that we want to create a lot of Date class instances having date information coming from an outer source encoded as a string with format "dd-mm-yyyy". Suppose we have to do this in different places in the source code of our project.

    So what we must do here is:

    1. Parse a string to receive day, month and year as three integer variables or a 3-item tuple consisting of that variable.
    2. Instantiate Date by passing those values to the initialization call.

    This will look like:

    day, month, year = map(int, string_date.split("-"))
    date1 = Date(day, month, year)
    

    For this purpose, C++ can implement such a feature with overloading, but Python lacks this overloading. Instead, we can use classmethod. Let"s create another "constructor".

        @classmethod
        def from_string(cls, date_as_string):
            day, month, year = map(int, date_as_string.split("-"))
            date1 = cls(day, month, year)
            return date1
    
    date2 = Date.from_string("11-09-2012")
    

    Let"s look more carefully at the above implementation, and review what advantages we have here:

    1. We"ve implemented date string parsing in one place and it"s reusable now.
    2. Encapsulation works fine here (if you think that you could implement string parsing as a single function elsewhere, this solution fits the OOP paradigm far better).
    3. cls is an object that holds the class itself, not an instance of the class. It"s pretty cool because if we inherit our Date class, all children will have from_string defined also.

    Static method

    What about staticmethod? It"s pretty similar to classmethod but doesn"t take any obligatory parameters (like a class method or instance method does).

    Let"s look at the next use case.

    We have a date string that we want to validate somehow. This task is also logically bound to the Date class we"ve used so far, but doesn"t require instantiation of it.

    Here is where staticmethod can be useful. Let"s look at the next piece of code:

        @staticmethod
        def is_date_valid(date_as_string):
            day, month, year = map(int, date_as_string.split("-"))
            return day <= 31 and month <= 12 and year <= 3999
    
        # usage:
        is_date = Date.is_date_valid("11-09-2012")
    

    So, as we can see from usage of staticmethod, we don"t have any access to what the class is---it"s basically just a function, called syntactically like a method, but without access to the object and its internals (fields and another methods), while classmethod does.

    Answer #2:

    Rostyslav Dzinko"s answer is very appropriate. I thought I could highlight one other reason you should choose @classmethod over @staticmethod when you are creating an additional constructor.

    In the example above, Rostyslav used the @classmethod from_string as a Factory to create Date objects from otherwise unacceptable parameters. The same can be done with @staticmethod as is shown in the code below:

    class Date:
      def __init__(self, month, day, year):
        self.month = month
        self.day   = day
        self.year  = year
    
    
      def display(self):
        return "{0}-{1}-{2}".format(self.month, self.day, self.year)
    
    
      @staticmethod
      def millenium(month, day):
        return Date(month, day, 2000)
    
    new_year = Date(1, 1, 2013)               # Creates a new Date object
    millenium_new_year = Date.millenium(1, 1) # also creates a Date object. 
    
    # Proof:
    new_year.display()           # "1-1-2013"
    millenium_new_year.display() # "1-1-2000"
    
    isinstance(new_year, Date) # True
    isinstance(millenium_new_year, Date) # True
    

    Thus both new_year and millenium_new_year are instances of the Date class.

    But, if you observe closely, the Factory process is hard-coded to create Date objects no matter what. What this means is that even if the Date class is subclassed, the subclasses will still create plain Date objects (without any properties of the subclass). See that in the example below:

    class DateTime(Date):
      def display(self):
          return "{0}-{1}-{2} - 00:00:00PM".format(self.month, self.day, self.year)
    
    
    datetime1 = DateTime(10, 10, 1990)
    datetime2 = DateTime.millenium(10, 10)
    
    isinstance(datetime1, DateTime) # True
    isinstance(datetime2, DateTime) # False
    
    datetime1.display() # returns "10-10-1990 - 00:00:00PM"
    datetime2.display() # returns "10-10-2000" because it"s not a DateTime object but a Date object. Check the implementation of the millenium method on the Date class for more details.
    

    datetime2 is not an instance of DateTime? WTF? Well, that"s because of the @staticmethod decorator used.

    In most cases, this is undesired. If what you want is a Factory method that is aware of the class that called it, then @classmethod is what you need.

    Rewriting Date.millenium as (that"s the only part of the above code that changes):

    @classmethod
    def millenium(cls, month, day):
        return cls(month, day, 2000)
    

    ensures that the class is not hard-coded but rather learnt. cls can be any subclass. The resulting object will rightly be an instance of cls.
    Let"s test that out:

    datetime1 = DateTime(10, 10, 1990)
    datetime2 = DateTime.millenium(10, 10)
    
    isinstance(datetime1, DateTime) # True
    isinstance(datetime2, DateTime) # True
    
    
    datetime1.display() # "10-10-1990 - 00:00:00PM"
    datetime2.display() # "10-10-2000 - 00:00:00PM"
    

    The reason is, as you know by now, that @classmethod was used instead of @staticmethod

    Answer #3:

    @classmethod means: when this method is called, we pass the class as the first argument instead of the instance of that class (as we normally do with methods). This means you can use the class and its properties inside that method rather than a particular instance.

    @staticmethod means: when this method is called, we don"t pass an instance of the class to it (as we normally do with methods). This means you can put a function inside a class but you can"t access the instance of that class (this is useful when your method does not use the instance).

    What is the meaning of single and double underscore before an object name?

    Can someone please explain the exact meaning of having single and double leading underscores before an object"s name in Python, and the difference between both?

    Also, does that meaning stay the same regardless of whether the object in question is a variable, a function, a method, etc.?

    Answer #1:

    Single Underscore

    Names, in a class, with a leading underscore are simply to indicate to other programmers that the attribute or method is intended to be private. However, nothing special is done with the name itself.

    To quote PEP-8:

    _single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.

    Double Underscore (Name Mangling)

    From the Python docs:

    Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.

    And a warning from the same page:

    Name mangling is intended to give classes an easy way to define “private” instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.

    Example

    >>> class MyClass():
    ...     def __init__(self):
    ...             self.__superprivate = "Hello"
    ...             self._semiprivate = ", world!"
    ...
    >>> mc = MyClass()
    >>> print mc.__superprivate
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: myClass instance has no attribute "__superprivate"
    >>> print mc._semiprivate
    , world!
    >>> print mc.__dict__
    {"_MyClass__superprivate": "Hello", "_semiprivate": ", world!"}
    

    Answer #2:

    __foo__: this is just a convention, a way for the Python system to use names that won"t conflict with user names.

    _foo: this is just a convention, a way for the programmer to indicate that the variable is private (whatever that means in Python).

    __foo: this has real meaning: the interpreter replaces this name with _classname__foo as a way to ensure that the name will not overlap with a similar name in another class.

    No other form of underscores have meaning in the Python world.

    There"s no difference between class, variable, global, etc in these conventions.

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    Finding median of list in Python

    How do you find the median of a list in Python? The list can be of any size and the numbers are not guaranteed to be in any particular order.

    If the list contains an even number of elements, the function should return the average of the middle two.

    Here are some examples (sorted for display purposes):

    median([1]) == 1
    median([1, 1]) == 1
    median([1, 1, 2, 4]) == 1.5
    median([0, 2, 5, 6, 8, 9, 9]) == 6
    median([0, 0, 0, 0, 4, 4, 6, 8]) == 2
    

    Answer #1:

    Python 3.4 has statistics.median:

    Return the median (middle value) of numeric data.

    When the number of data points is odd, return the middle data point. When the number of data points is even, the median is interpolated by taking the average of the two middle values:

    >>> median([1, 3, 5])
    3
    >>> median([1, 3, 5, 7])
    4.0
    

    Usage:

    import statistics
    
    items = [6, 1, 8, 2, 3]
    
    statistics.median(items)
    #>>> 3
    

    It"s pretty careful with types, too:

    statistics.median(map(float, items))
    #>>> 3.0
    
    from decimal import Decimal
    statistics.median(map(Decimal, items))
    #>>> Decimal("3")
    

    Answer #2:

    (Works with ):

    def median(lst):
        n = len(lst)
        s = sorted(lst)
        return (sum(s[n//2-1:n//2+1])/2.0, s[n//2])[n % 2] if n else None
    

    >>> median([-5, -5, -3, -4, 0, -1])
    -3.5
    

    numpy.median():

    >>> from numpy import median
    >>> median([1, -4, -1, -1, 1, -3])
    -1.0
    

    For , use statistics.median:

    >>> from statistics import median
    >>> median([5, 2, 3, 8, 9, -2])
    4.0
    

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    How can I open multiple files using "with open" in Python?

    I want to change a couple of files at one time, iff I can write to all of them. I"m wondering if I somehow can combine the multiple open calls with the with statement:

    try:
      with open("a", "w") as a and open("b", "w") as b:
        do_something()
    except IOError as e:
      print "Operation failed: %s" % e.strerror
    

    If that"s not possible, what would an elegant solution to this problem look like?

    Answer #1:

    As of Python 2.7 (or 3.1 respectively) you can write

    with open("a", "w") as a, open("b", "w") as b:
        do_something()
    

    In earlier versions of Python, you can sometimes use contextlib.nested() to nest context managers. This won"t work as expected for opening multiples files, though -- see the linked documentation for details.


    In the rare case that you want to open a variable number of files all at the same time, you can use contextlib.ExitStack, starting from Python version 3.3:

    with ExitStack() as stack:
        files = [stack.enter_context(open(fname)) for fname in filenames]
        # Do something with "files"
    

    Most of the time you have a variable set of files, you likely want to open them one after the other, though.

    Answer #2:

    For opening many files at once or for long file paths, it may be useful to break things up over multiple lines. From the Python Style Guide as suggested by @Sven Marnach in comments to another answer:

    with open("/path/to/InFile.ext", "r") as file_1, 
         open("/path/to/OutFile.ext", "w") as file_2:
        file_2.write(file_1.read())
    

    open() in Python does not create a file if it doesn"t exist

    What is the best way to open a file as read/write if it exists, or if it does not, then create it and open it as read/write? From what I read, file = open("myfile.dat", "rw") should do this, right?

    It is not working for me (Python 2.6.2) and I"m wondering if it is a version problem, or not supposed to work like that or what.

    The bottom line is, I just need a solution for the problem. I am curious about the other stuff, but all I need is a nice way to do the opening part.

    The enclosing directory was writeable by user and group, not other (I"m on a Linux system... so permissions 775 in other words), and the exact error was:

    IOError: no such file or directory.

    Answer #1:

    You should use open with the w+ mode:

    file = open("myfile.dat", "w+")
    

    Answer #2:

    The advantage of the following approach is that the file is properly closed at the block"s end, even if an exception is raised on the way. It"s equivalent to try-finally, but much shorter.

    with open("file.dat";"a+") as f:
        f.write(...)
        ...
    

    a+ Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing. -Python file modes

    seek() method sets the file"s current position.

    f.seek(pos [, (0|1|2)])
    pos .. position of the r/w pointer
    [] .. optionally
    () .. one of ->
      0 .. absolute position
      1 .. relative position to current
      2 .. relative position from end
    

    Only "rwab+" characters are allowed; there must be exactly one of "rwa" - see Stack Overflow question Python file modes detail.

    Difference between modes a, a+, w, w+, and r+ in built-in open function?

    In the python built-in open function, what is the exact difference between the modes w, a, w+, a+, and r+?

    In particular, the documentation implies that all of these will allow writing to the file, and says that it opens the files for "appending", "writing", and "updating" specifically, but does not define what these terms mean.

    Answer #1:

    The opening modes are exactly the same as those for the C standard library function fopen().

    The BSD fopen manpage defines them as follows:

     The argument mode points to a string beginning with one of the following
     sequences (Additional characters may follow these sequences.):
    
     ``r""   Open text file for reading.  The stream is positioned at the
             beginning of the file.
    
     ``r+""  Open for reading and writing.  The stream is positioned at the
             beginning of the file.
    
     ``w""   Truncate file to zero length or create text file for writing.
             The stream is positioned at the beginning of the file.
    
     ``w+""  Open for reading and writing.  The file is created if it does not
             exist, otherwise it is truncated.  The stream is positioned at
             the beginning of the file.
    
     ``a""   Open for writing.  The file is created if it does not exist.  The
             stream is positioned at the end of the file.  Subsequent writes
             to the file will always end up at the then current end of file,
             irrespective of any intervening fseek(3) or similar.
    
     ``a+""  Open for reading and writing.  The file is created if it does not
             exist.  The stream is positioned at the end of the file.  Subse-
             quent writes to the file will always end up at the then current
             end of file, irrespective of any intervening fseek(3) or similar.
    

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    How do I merge two dictionaries in a single expression (taking union of dictionaries)?

    Question by Carl Meyer

    I have two Python dictionaries, and I want to write a single expression that returns these two dictionaries, merged (i.e. taking the union). The update() method would be what I need, if it returned its result instead of modifying a dictionary in-place.

    >>> x = {"a": 1, "b": 2}
    >>> y = {"b": 10, "c": 11}
    >>> z = x.update(y)
    >>> print(z)
    None
    >>> x
    {"a": 1, "b": 10, "c": 11}
    

    How can I get that final merged dictionary in z, not x?

    (To be extra-clear, the last-one-wins conflict-handling of dict.update() is what I"m looking for as well.)

    Answer #1:

    How can I merge two Python dictionaries in a single expression?

    For dictionaries x and y, z becomes a shallowly-merged dictionary with values from y replacing those from x.

    • In Python 3.9.0 or greater (released 17 October 2020): PEP-584, discussed here, was implemented and provides the simplest method:

      z = x | y          # NOTE: 3.9+ ONLY
      
    • In Python 3.5 or greater:

      z = {**x, **y}
      
    • In Python 2, (or 3.4 or lower) write a function:

      def merge_two_dicts(x, y):
          z = x.copy()   # start with keys and values of x
          z.update(y)    # modifies z with keys and values of y
          return z
      

      and now:

      z = merge_two_dicts(x, y)
      

    Explanation

    Say you have two dictionaries and you want to merge them into a new dictionary without altering the original dictionaries:

    x = {"a": 1, "b": 2}
    y = {"b": 3, "c": 4}
    

    The desired result is to get a new dictionary (z) with the values merged, and the second dictionary"s values overwriting those from the first.

    >>> z
    {"a": 1, "b": 3, "c": 4}
    

    A new syntax for this, proposed in PEP 448 and available as of Python 3.5, is

    z = {**x, **y}
    

    And it is indeed a single expression.

    Note that we can merge in with literal notation as well:

    z = {**x, "foo": 1, "bar": 2, **y}
    

    and now:

    >>> z
    {"a": 1, "b": 3, "foo": 1, "bar": 2, "c": 4}
    

    It is now showing as implemented in the release schedule for 3.5, PEP 478, and it has now made its way into the What"s New in Python 3.5 document.

    However, since many organizations are still on Python 2, you may wish to do this in a backward-compatible way. The classically Pythonic way, available in Python 2 and Python 3.0-3.4, is to do this as a two-step process:

    z = x.copy()
    z.update(y) # which returns None since it mutates z
    

    In both approaches, y will come second and its values will replace x"s values, thus b will point to 3 in our final result.

    Not yet on Python 3.5, but want a single expression

    If you are not yet on Python 3.5 or need to write backward-compatible code, and you want this in a single expression, the most performant while the correct approach is to put it in a function:

    def merge_two_dicts(x, y):
        """Given two dictionaries, merge them into a new dict as a shallow copy."""
        z = x.copy()
        z.update(y)
        return z
    

    and then you have a single expression:

    z = merge_two_dicts(x, y)
    

    You can also make a function to merge an arbitrary number of dictionaries, from zero to a very large number:

    def merge_dicts(*dict_args):
        """
        Given any number of dictionaries, shallow copy and merge into a new dict,
        precedence goes to key-value pairs in latter dictionaries.
        """
        result = {}
        for dictionary in dict_args:
            result.update(dictionary)
        return result
    

    This function will work in Python 2 and 3 for all dictionaries. e.g. given dictionaries a to g:

    z = merge_dicts(a, b, c, d, e, f, g) 
    

    and key-value pairs in g will take precedence over dictionaries a to f, and so on.

    Critiques of Other Answers

    Don"t use what you see in the formerly accepted answer:

    z = dict(x.items() + y.items())
    

    In Python 2, you create two lists in memory for each dict, create a third list in memory with length equal to the length of the first two put together, and then discard all three lists to create the dict. In Python 3, this will fail because you"re adding two dict_items objects together, not two lists -

    >>> c = dict(a.items() + b.items())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: unsupported operand type(s) for +: "dict_items" and "dict_items"
    

    and you would have to explicitly create them as lists, e.g. z = dict(list(x.items()) + list(y.items())). This is a waste of resources and computation power.

    Similarly, taking the union of items() in Python 3 (viewitems() in Python 2.7) will also fail when values are unhashable objects (like lists, for example). Even if your values are hashable, since sets are semantically unordered, the behavior is undefined in regards to precedence. So don"t do this:

    >>> c = dict(a.items() | b.items())
    

    This example demonstrates what happens when values are unhashable:

    >>> x = {"a": []}
    >>> y = {"b": []}
    >>> dict(x.items() | y.items())
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: unhashable type: "list"
    

    Here"s an example where y should have precedence, but instead the value from x is retained due to the arbitrary order of sets:

    >>> x = {"a": 2}
    >>> y = {"a": 1}
    >>> dict(x.items() | y.items())
    {"a": 2}
    

    Another hack you should not use:

    z = dict(x, **y)
    

    This uses the dict constructor and is very fast and memory-efficient (even slightly more so than our two-step process) but unless you know precisely what is happening here (that is, the second dict is being passed as keyword arguments to the dict constructor), it"s difficult to read, it"s not the intended usage, and so it is not Pythonic.

    Here"s an example of the usage being remediated in django.

    Dictionaries are intended to take hashable keys (e.g. frozensets or tuples), but this method fails in Python 3 when keys are not strings.

    >>> c = dict(a, **b)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: keyword arguments must be strings
    

    From the mailing list, Guido van Rossum, the creator of the language, wrote:

    I am fine with declaring dict({}, **{1:3}) illegal, since after all it is abuse of the ** mechanism.

    and

    Apparently dict(x, **y) is going around as "cool hack" for "call x.update(y) and return x". Personally, I find it more despicable than cool.

    It is my understanding (as well as the understanding of the creator of the language) that the intended usage for dict(**y) is for creating dictionaries for readability purposes, e.g.:

    dict(a=1, b=10, c=11)
    

    instead of

    {"a": 1, "b": 10, "c": 11}
    

    Response to comments

    Despite what Guido says, dict(x, **y) is in line with the dict specification, which btw. works for both Python 2 and 3. The fact that this only works for string keys is a direct consequence of how keyword parameters work and not a short-coming of dict. Nor is using the ** operator in this place an abuse of the mechanism, in fact, ** was designed precisely to pass dictionaries as keywords.

    Again, it doesn"t work for 3 when keys are not strings. The implicit calling contract is that namespaces take ordinary dictionaries, while users must only pass keyword arguments that are strings. All other callables enforced it. dict broke this consistency in Python 2:

    >>> foo(**{("a", "b"): None})
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: foo() keywords must be strings
    >>> dict(**{("a", "b"): None})
    {("a", "b"): None}
    

    This inconsistency was bad given other implementations of Python (PyPy, Jython, IronPython). Thus it was fixed in Python 3, as this usage could be a breaking change.

    I submit to you that it is malicious incompetence to intentionally write code that only works in one version of a language or that only works given certain arbitrary constraints.

    More comments:

    dict(x.items() + y.items()) is still the most readable solution for Python 2. Readability counts.

    My response: merge_two_dicts(x, y) actually seems much clearer to me, if we"re actually concerned about readability. And it is not forward compatible, as Python 2 is increasingly deprecated.

    {**x, **y} does not seem to handle nested dictionaries. the contents of nested keys are simply overwritten, not merged [...] I ended up being burnt by these answers that do not merge recursively and I was surprised no one mentioned it. In my interpretation of the word "merging" these answers describe "updating one dict with another", and not merging.

    Yes. I must refer you back to the question, which is asking for a shallow merge of two dictionaries, with the first"s values being overwritten by the second"s - in a single expression.

    Assuming two dictionaries of dictionaries, one might recursively merge them in a single function, but you should be careful not to modify the dictionaries from either source, and the surest way to avoid that is to make a copy when assigning values. As keys must be hashable and are usually therefore immutable, it is pointless to copy them:

    from copy import deepcopy
    
    def dict_of_dicts_merge(x, y):
        z = {}
        overlapping_keys = x.keys() & y.keys()
        for key in overlapping_keys:
            z[key] = dict_of_dicts_merge(x[key], y[key])
        for key in x.keys() - overlapping_keys:
            z[key] = deepcopy(x[key])
        for key in y.keys() - overlapping_keys:
            z[key] = deepcopy(y[key])
        return z
    

    Usage:

    >>> x = {"a":{1:{}}, "b": {2:{}}}
    >>> y = {"b":{10:{}}, "c": {11:{}}}
    >>> dict_of_dicts_merge(x, y)
    {"b": {2: {}, 10: {}}, "a": {1: {}}, "c": {11: {}}}
    

    Coming up with contingencies for other value types is far beyond the scope of this question, so I will point you at my answer to the canonical question on a "Dictionaries of dictionaries merge".

    Less Performant But Correct Ad-hocs

    These approaches are less performant, but they will provide correct behavior. They will be much less performant than copy and update or the new unpacking because they iterate through each key-value pair at a higher level of abstraction, but they do respect the order of precedence (latter dictionaries have precedence)

    You can also chain the dictionaries manually inside a dict comprehension:

    {k: v for d in dicts for k, v in d.items()} # iteritems in Python 2.7
    

    or in Python 2.6 (and perhaps as early as 2.4 when generator expressions were introduced):

    dict((k, v) for d in dicts for k, v in d.items()) # iteritems in Python 2
    

    itertools.chain will chain the iterators over the key-value pairs in the correct order:

    from itertools import chain
    z = dict(chain(x.items(), y.items())) # iteritems in Python 2
    

    Performance Analysis

    I"m only going to do the performance analysis of the usages known to behave correctly. (Self-contained so you can copy and paste yourself.)

    from timeit import repeat
    from itertools import chain
    
    x = dict.fromkeys("abcdefg")
    y = dict.fromkeys("efghijk")
    
    def merge_two_dicts(x, y):
        z = x.copy()
        z.update(y)
        return z
    
    min(repeat(lambda: {**x, **y}))
    min(repeat(lambda: merge_two_dicts(x, y)))
    min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
    min(repeat(lambda: dict(chain(x.items(), y.items()))))
    min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
    

    In Python 3.8.1, NixOS:

    >>> min(repeat(lambda: {**x, **y}))
    1.0804965235292912
    >>> min(repeat(lambda: merge_two_dicts(x, y)))
    1.636518670246005
    >>> min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
    3.1779992282390594
    >>> min(repeat(lambda: dict(chain(x.items(), y.items()))))
    2.740647904574871
    >>> min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
    4.266070580109954
    
    $ uname -a
    Linux nixos 4.19.113 #1-NixOS SMP Wed Mar 25 07:06:15 UTC 2020 x86_64 GNU/Linux
    

    Resources on Dictionaries

    Answer #2:

    In your case, what you can do is:

    z = dict(list(x.items()) + list(y.items()))
    

    This will, as you want it, put the final dict in z, and make the value for key b be properly overridden by the second (y) dict"s value:

    >>> x = {"a":1, "b": 2}
    >>> y = {"b":10, "c": 11}
    >>> z = dict(list(x.items()) + list(y.items()))
    >>> z
    {"a": 1, "c": 11, "b": 10}
    
    

    If you use Python 2, you can even remove the list() calls. To create z:

    >>> z = dict(x.items() + y.items())
    >>> z
    {"a": 1, "c": 11, "b": 10}
    

    If you use Python version 3.9.0a4 or greater, then you can directly use:

    x = {"a":1, "b": 2}
    y = {"b":10, "c": 11}
    z = x | y
    print(z)
    
    {"a": 1, "c": 11, "b": 10}
    

    Answer #3:

    An alternative:

    z = x.copy()
    z.update(y)
    

    Answer #4:

    Another, more concise, option:

    z = dict(x, **y)
    

    Note: this has become a popular answer, but it is important to point out that if y has any non-string keys, the fact that this works at all is an abuse of a CPython implementation detail, and it does not work in Python 3, or in PyPy, IronPython, or Jython. Also, Guido is not a fan. So I can"t recommend this technique for forward-compatible or cross-implementation portable code, which really means it should be avoided entirely.

    Answer #5:

    This probably won"t be a popular answer, but you almost certainly do not want to do this. If you want a copy that"s a merge, then use copy (or deepcopy, depending on what you want) and then update. The two lines of code are much more readable - more Pythonic - than the single line creation with .items() + .items(). Explicit is better than implicit.

    In addition, when you use .items() (pre Python 3.0), you"re creating a new list that contains the items from the dict. If your dictionaries are large, then that is quite a lot of overhead (two large lists that will be thrown away as soon as the merged dict is created). update() can work more efficiently, because it can run through the second dict item-by-item.

    In terms of time:

    >>> timeit.Timer("dict(x, **y)", "x = dict(zip(range(1000), range(1000)))
    y=dict(zip(range(1000,2000), range(1000,2000)))").timeit(100000)
    15.52571702003479
    >>> timeit.Timer("temp = x.copy()
    temp.update(y)", "x = dict(zip(range(1000), range(1000)))
    y=dict(zip(range(1000,2000), range(1000,2000)))").timeit(100000)
    15.694622993469238
    >>> timeit.Timer("dict(x.items() + y.items())", "x = dict(zip(range(1000), range(1000)))
    y=dict(zip(range(1000,2000), range(1000,2000)))").timeit(100000)
    41.484580039978027
    

    IMO the tiny slowdown between the first two is worth it for the readability. In addition, keyword arguments for dictionary creation was only added in Python 2.3, whereas copy() and update() will work in older versions.

    How to recursively delete a directory and all its contents (files + subdirectories) in PHP?: StackOverflow Questions

    How to execute a program or call a system command?

    Question by alan lai

    How do you call an external command (as if I"d typed it at the Unix shell or Windows command prompt) from within a Python script?

    Answer #1:

    Use the subprocess module in the standard library:

    import subprocess
    subprocess.run(["ls", "-l"])
    

    The advantage of subprocess.run over os.system is that it is more flexible (you can get the stdout, stderr, the "real" status code, better error handling, etc...).

    Even the documentation for os.system recommends using subprocess instead:

    The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes.

    On Python 3.4 and earlier, use subprocess.call instead of .run:

    subprocess.call(["ls", "-l"])
    

    Answer #2:

    Here"s a summary of the ways to call external programs and the advantages and disadvantages of each:

    1. os.system("some_command with args") passes the command and arguments to your system"s shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example:

      os.system("some_command < input_file | another_command > output_file")  
      

      However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, et cetera. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs. See the documentation.

    2. stream = os.popen("some_command with args") will do the same thing as os.system except that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don"t need to worry about escaping anything. See the documentation.

    3. The Popen class of the subprocess module. This is intended as a replacement for os.popen, but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you"d say:

      print subprocess.Popen("echo Hello World", shell=True, stdout=subprocess.PIPE).stdout.read()
      

      instead of

      print os.popen("echo Hello World").read()
      

      but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See the documentation.

    4. The call function from the subprocess module. This is basically just like the Popen class and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example:

      return_code = subprocess.call("echo Hello World", shell=True)
      

      See the documentation.

    5. If you"re on Python 3.5 or later, you can use the new subprocess.run function, which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing.

    6. The os module also has all of the fork/exec/spawn functions that you"d have in a C program, but I don"t recommend using them directly.

    The subprocess module should probably be what you use.

    Finally, please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. There are serious security implications if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications consider this code:

    print subprocess.Popen("echo %s " % user_input, stdout=PIPE).stdout.read()
    

    and imagine that the user enters something "my mama didnt love me && rm -rf /" which could erase the whole filesystem.

    Answer #3:

    Typical implementation:

    import subprocess
    
    p = subprocess.Popen("ls", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in p.stdout.readlines():
        print line,
    retval = p.wait()
    

    You are free to do what you want with the stdout data in the pipe. In fact, you can simply omit those parameters (stdout= and stderr=) and it"ll behave like os.system().

    Answer #4:

    Some hints on detaching the child process from the calling one (starting the child process in background).

    Suppose you want to start a long task from a CGI script. That is, the child process should live longer than the CGI script execution process.

    The classical example from the subprocess module documentation is:

    import subprocess
    import sys
    
    # Some code here
    
    pid = subprocess.Popen([sys.executable, "longtask.py"]) # Call subprocess
    
    # Some more code here
    

    The idea here is that you do not want to wait in the line "call subprocess" until the longtask.py is finished. But it is not clear what happens after the line "some more code here" from the example.

    My target platform was FreeBSD, but the development was on Windows, so I faced the problem on Windows first.

    On Windows (Windows XP), the parent process will not finish until the longtask.py has finished its work. It is not what you want in a CGI script. The problem is not specific to Python; in the PHP community the problems are the same.

    The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in Windows API. If you happen to have installed pywin32, you can import the flag from the win32process module, otherwise you should define it yourself:

    DETACHED_PROCESS = 0x00000008
    
    pid = subprocess.Popen([sys.executable, "longtask.py"],
                           creationflags=DETACHED_PROCESS).pid
    

    /* UPD 2015.10.27 @eryksun in a comment below notes, that the semantically correct flag is CREATE_NEW_CONSOLE (0x00000010) */

    On FreeBSD we have another problem: when the parent process is finished, it finishes the child processes as well. And that is not what you want in a CGI script either. Some experiments showed that the problem seemed to be in sharing sys.stdout. And the working solution was the following:

    pid = subprocess.Popen([sys.executable, "longtask.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
    

    I have not checked the code on other platforms and do not know the reasons of the behaviour on FreeBSD. If anyone knows, please share your ideas. Googling on starting background processes in Python does not shed any light yet.

    Answer #5:

    import os
    os.system("your command")
    

    Note that this is dangerous, since the command isn"t cleaned. I leave it up to you to google for the relevant documentation on the "os" and "sys" modules. There are a bunch of functions (exec* and spawn*) that will do similar things.

Shop

Best laptop for Sims 4

$

Best laptop for Zoom

$499

Best laptop for Minecraft

$590

Best laptop for engineering student

$

Best laptop for development

$

Best laptop for Cricut Maker

$

Best laptop for hacking

$890

Best laptop for Machine Learning

$950

Latest questions

NUMPYNUMPY

psycopg2: insert multiple rows with one query

12 answers

NUMPYNUMPY

How to convert Nonetype to int or string?

12 answers

NUMPYNUMPY

How to specify multiple return types using type-hints

12 answers

NUMPYNUMPY

Javascript Error: IPython is not defined in JupyterLab

12 answers

Wiki

Python OpenCV | cv2.putText () method

numpy.arctan2 () in Python

Python | os.path.realpath () method

Python OpenCV | cv2.circle () method

Python OpenCV cv2.cvtColor () method

Python - Move item to the end of the list

time.perf_counter () function in Python

Check if one list is a subset of another in Python

Python os.path.join () method