👻 Check our latest review to choose the best laptop for Machine Learning engineers and Deep learning tasks!
I"m having much trouble trying to understand just how the multiprocessing queue works on python and how to implement it. Lets say I have two python modules that access data from a shared file, let"s call these two modules a writer and a reader. My plan is to have both the reader and writer put requests into two separate multiprocessing queues, and then have a third process pop these requests in a loop and execute as such.
My main problem is that I really don"t know how to implement multiprocessing.queue correctly, you cannot really instantiate the object for each process since they will be separate queues, how do you make sure that all processes relate to a shared queue (or in this case, queues)
👻 Read also: what is the best laptop for engineering students?
How to use multiprocessing queue in Python? iat: Questions
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately
3 answers
Tried to perform REST GET through python requests with the following code and I got error.
Code snip:
import requests
header = {"Authorization": "Bearer..."}
url = az_base_url + az_subscription_id + "/resourcegroups/Default-Networking/resources?" + az_api_version
r = requests.get(url, headers=header)
Error:
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:79:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail.
For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
My python version is 2.7.3. I tried to install urllib3 and requests[security] as some other thread suggests, I still got the same error.
Wonder if anyone can provide some tips?
Answer #1
The docs give a fair indicator of what"s required., however requests allow us to skip a few steps:
You only need to install the security package extras (thanks @admdrew for pointing it out)
$ pip install requests[security]
or, install them directly:
$ pip install pyopenssl ndg-httpsclient pyasn1
Requests will then automatically inject pyopenssl into urllib3
If you"re on ubuntu, you may run into trouble installing pyopenssl, you"ll need these dependencies:
$ apt-get install libffi-dev libssl-dev
How to use multiprocessing queue in Python? iat: Questions
Dynamic instantiation from string name of a class in dynamically imported module?
3 answers
In python, I have to instantiate certain class, knowing its name in a string, but this class "lives" in a dynamically imported module. An example follows:
loader-class script:
import sys
class loader:
def __init__(self, module_name, class_name): # both args are strings
try:
__import__(module_name)
modul = sys.modules[module_name]
instance = modul.class_name() # obviously this doesn"t works, here is my main problem!
except ImportError:
# manage import error
some-dynamically-loaded-module script:
class myName:
# etc...
I use this arrangement to make any dynamically-loaded-module to be used by the loader-class following certain predefined behaviours in the dyn-loaded-modules...
Answer #1
You can use getattr
getattr(module, class_name)
to access the class. More complete code:
module = __import__(module_name)
class_ = getattr(module, class_name)
instance = class_()
As mentioned below, we may use importlib
import importlib
module = importlib.import_module(module_name)
class_ = getattr(module, class_name)
instance = class_()
How to use multiprocessing queue in Python? iat: Questions
How to get all of the immediate subdirectories in Python
3 answers
I"m trying to write a simple Python script that will copy a index.tpl to index.html in all of the subdirectories (with a few exceptions).
I"m getting bogged down by trying to get the list of subdirectories.
Answer #1
import os
def get_immediate_subdirectories(a_dir):
return [name for name in os.listdir(a_dir)
if os.path.isdir(os.path.join(a_dir, name))]
How to print number with commas as thousands separators?
5 answers
I am trying to print an integer in Python 2.6.1 with commas as thousands separators. For example, I want to show the number 1234567 as 1,234,567. How would I go about doing this? I have seen many examples on Google, but I am looking for the simplest practical way.
It does not need to be locale-specific to decide between periods and commas. I would prefer something as simple as reasonably possible.
Answer #1
Locale unaware
"{:,}".format(value) # For Python ‚â•2.7
f"{value:,}" # For Python ‚â•3.6
Locale aware
import locale
locale.setlocale(locale.LC_ALL, "") # Use "" for auto, or force e.g. to "en_US.UTF-8"
"{:n}".format(value) # For Python ‚â•2.7
f"{value:n}" # For Python ‚â•3.6
Reference
Per Format Specification Mini-Language,
The
","option signals the use of a comma for a thousands separator. For a locale aware separator, use the"n"integer presentation type instead.
Answer #2
I got this to work:
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "en_US")
"en_US"
>>> locale.format("%d", 1255000, grouping=True)
"1,255,000"
Sure, you don"t need internationalization support, but it"s clear, concise, and uses a built-in library.
P.S. That "%d" is the usual %-style formatter. You can have only one formatter, but it can be whatever you need in terms of field width and precision settings.
P.P.S. If you can"t get locale to work, I"d suggest a modified version of Mark"s answer:
def intWithCommas(x):
if type(x) not in [type(0), type(0L)]:
raise TypeError("Parameter must be an integer.")
if x < 0:
return "-" + intWithCommas(-x)
result = ""
while x >= 1000:
x, r = divmod(x, 1000)
result = ",%03d%s" % (r, result)
return "%d%s" % (x, result)
Recursion is useful for the negative case, but one recursion per comma seems a bit excessive to me.
How would you make a comma-separated string from a list of strings?
5 answers
What would be your preferred way to concatenate strings from a sequence such that between every two consecutive pairs a comma is added. That is, how do you map, for instance, ["a", "b", "c"] to "a,b,c"? (The cases ["s"] and [] should be mapped to "s" and "", respectively.)
I usually end up using something like "".join(map(lambda x: x+",",l))[:-1], but also feeling somewhat unsatisfied.
Answer #1
my_list = ["a", "b", "c", "d"]
my_string = ",".join(my_list)
"a,b,c,d"
This won"t work if the list contains integers
And if the list contains non-string types (such as integers, floats, bools, None) then do:
my_string = ",".join(map(str, my_list))
We hope this article has helped you to resolve the problem. Apart from How to use multiprocessing queue in Python?, check other iat-related topics.
Want to excel in Python? See our review of the best Python online courses 2022. If you are interested in Data Science, check also how to learn programming in R.
By the way, this material is also available in other languages:
- Italiano How to use multiprocessing queue in Python?
- Deutsch How to use multiprocessing queue in Python?
- Français How to use multiprocessing queue in Python?
- Español How to use multiprocessing queue in Python?
- Türk How to use multiprocessing queue in Python?
- Русский How to use multiprocessing queue in Python?
- Português How to use multiprocessing queue in Python?
- Polski How to use multiprocessing queue in Python?
- Nederlandse How to use multiprocessing queue in Python?
- 中文 How to use multiprocessing queue in Python?
- 한국어 How to use multiprocessing queue in Python?
- 日本語 How to use multiprocessing queue in Python?
- हिन्दी How to use multiprocessing queue in Python?
Boston | 2022-11-28
Simply put and clear. Thank you for sharing. How to use multiprocessing queue in Python? and other issues with StackOverflow was always my weak point 😁. Will get back tomorrow with feedback
Berlin | 2022-11-28
I was preparing for my coding interview, thanks for clarifying this - How to use multiprocessing queue in Python? in Python is not the simplest one. I am just not quite sure it is the best method
London | 2022-11-28
Thanks for explaining! I was stuck with How to use multiprocessing queue in Python? for some hours, finally got it done 🤗. Will use it in my bachelor thesis
