Why does multiprocessing use only a single core after I import numpy?

👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!

I am not sure whether this counts more as an OS issue, but I thought I would ask here in case anyone has some insight from the Python end of things.

I"ve been trying to parallelise a CPU-heavy for loop using joblib, but I find that instead of each worker process being assigned to a different core, I end up with all of them being assigned to the same core and no performance gain.

Here"s a very trivial example...

from joblib import Parallel,delayed
import numpy as np

def testfunc(data):
    # some very boneheaded CPU work
    for nn in xrange(1000):
        for ii in data[0,:]:
            for jj in data[1,:]:
                ii*jj

def run(niter=10):
    data = (np.random.randn(2,100) for ii in xrange(niter))
    pool = Parallel(n_jobs=-1,verbose=1,pre_dispatch="all")
    results = pool(delayed(testfunc)(dd) for dd in data)

if __name__ == "__main__":
    run()

...and here"s what I see in htop while this script is running:

I"m running Ubuntu 12.10 (3.5.0-26) on a laptop with 4 cores. Clearly joblib.Parallel is spawning separate processes for the different workers, but is there any way that I can make these processes execute on different cores?

👻 Read also: what is the best laptop for engineering students?

Why does multiprocessing use only a single core after I import numpy? __del__: Questions

import time
time.sleep(5)   # Delays for 5 seconds. You can also use a float value.

import time
while True:
    print("This prints once a minute.")
    time.sleep(60) # Delay for 1 minute (60 seconds).

from time import sleep
sleep(0.1) # Time in seconds

Why does multiprocessing use only a single core after I import numpy? __del__: Questions

We hope this article has helped you to resolve the problem. Apart from Why does multiprocessing use only a single core after I import numpy?, check other __del__-related topics.

Want to excel in Python? See our review of the best Python online courses 2023. If you are interested in Data Science, check also how to learn programming in R.

By the way, this material is also available in other languages:

• Italiano Why does multiprocessing use only a single core after I import numpy?
• Deutsch Why does multiprocessing use only a single core after I import numpy?
• Français Why does multiprocessing use only a single core after I import numpy?
• Español Why does multiprocessing use only a single core after I import numpy?
• Türk Why does multiprocessing use only a single core after I import numpy?
• Русский Why does multiprocessing use only a single core after I import numpy?
• Português Why does multiprocessing use only a single core after I import numpy?
• Polski Why does multiprocessing use only a single core after I import numpy?
• Nederlandse Why does multiprocessing use only a single core after I import numpy?
• 中文 Why does multiprocessing use only a single core after I import numpy?
• 한국어 Why does multiprocessing use only a single core after I import numpy?
• 日本語 Why does multiprocessing use only a single core after I import numpy?
• हिन्दी Why does multiprocessing use only a single core after I import numpy?