👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!
The documentation for the multiprocessing
module shows how to pass a queue to a process started with multiprocessing.Process
. But how can I share a queue with asynchronous worker processes started with apply_async
? I don"t need dynamic joining or anything else, just a way for the workers to (repeatedly) report their results back to base.
import multiprocessing
def worker(name, que):
que.put("%d is done" % name)
if __name__ == "__main__":
pool = multiprocessing.Pool(processes=3)
q = multiprocessing.Queue()
workers = pool.apply_async(worker, (33, q))
This fails with:
RuntimeError: Queue objects should only be shared between processes through inheritance
.
I understand what this means, and I understand the advice to inherit rather than require pickling/unpickling (and all the special Windows restrictions). But how do I pass the queue in a way that works? I can"t find an example, and I"ve tried several alternatives that failed in various ways. Help please?
👻 Read also: what is the best laptop for engineering students?
We hope this article has helped you to resolve the problem. Apart from Sharing a result queue among several processes, check other __main__ Python module-related topics.
Want to excel in Python? See our review of the best Python online courses 2023. If you are interested in Data Science, check also how to learn programming in R.
By the way, this material is also available in other languages:
- Italiano Sharing a result queue among several processes
- Deutsch Sharing a result queue among several processes
- Français Sharing a result queue among several processes
- Español Sharing a result queue among several processes
- Türk Sharing a result queue among several processes
- Русский Sharing a result queue among several processes
- Português Sharing a result queue among several processes
- Polski Sharing a result queue among several processes
- Nederlandse Sharing a result queue among several processes
- 中文 Sharing a result queue among several processes
- 한국어 Sharing a result queue among several processes
- 日本語 Sharing a result queue among several processes
- हिन्दी Sharing a result queue among several processes
Tallinn | 2023-03-26
Thanks for explaining! I was stuck with Sharing a result queue among several processes for some hours, finally got it done 🤗. Checked yesterday, it works!
Paris | 2023-03-26
Thanks for explaining! I was stuck with Sharing a result queue among several processes for some hours, finally got it done 🤗. I am just not quite sure it is the best method
Singapore | 2023-03-26
I was preparing for my coding interview, thanks for clarifying this - Sharing a result queue among several processes in Python is not the simplest one. Checked yesterday, it works!