I"m having much trouble trying to understand just how the multiprocessing queue works on python and how to implement it. Lets say I have two python modules that access data from a shared file, let"s call these two modules a writer and a reader. My plan is to have both the reader and writer put requests into two separate multiprocessing queues, and then have a third process pop these requests in a loop and execute as such.
My main problem is that I really don"t know how to implement multiprocessing.queue correctly, you cannot really instantiate the object for each process since they will be separate queues, how do you make sure that all processes relate to a shared queue (or in this case, queues)
How to use multiprocessing queue in Python? iat: Questions
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately
3 answers
Tried to perform REST GET through python requests with the following code and I got error.
Code snip:
import requests
header = {"Authorization": "Bearer..."}
url = az_base_url + az_subscription_id + "/resourcegroups/Default-Networking/resources?" + az_api_version
r = requests.get(url, headers=header)
Error:
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:79:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail.
For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
My python version is 2.7.3. I tried to install urllib3 and requests[security] as some other thread suggests, I still got the same error.
Wonder if anyone can provide some tips?
Answer #1
The docs give a fair indicator of what"s required., however requests allow us to skip a few steps:
You only need to install the security package extras (thanks @admdrew for pointing it out)
$ pip install requests[security]
or, install them directly:
$ pip install pyopenssl ndg-httpsclient pyasn1
Requests will then automatically inject pyopenssl into urllib3
If you"re on ubuntu, you may run into trouble installing pyopenssl, you"ll need these dependencies:
$ apt-get install libffi-dev libssl-dev
Dynamic instantiation from string name of a class in dynamically imported module?
3 answers
In python, I have to instantiate certain class, knowing its name in a string, but this class "lives" in a dynamically imported module. An example follows:
loader-class script:
import sys
class loader:
def __init__(self, module_name, class_name): # both args are strings
try:
__import__(module_name)
modul = sys.modules[module_name]
instance = modul.class_name() # obviously this doesn"t works, here is my main problem!
except ImportError:
# manage import error
some-dynamically-loaded-module script:
class myName:
# etc...
I use this arrangement to make any dynamically-loaded-module to be used by the loader-class following certain predefined behaviours in the dyn-loaded-modules...
Answer #1
You can use getattr
getattr(module, class_name)
to access the class. More complete code:
module = __import__(module_name)
class_ = getattr(module, class_name)
instance = class_()
As mentioned below, we may use importlib
import importlib
module = importlib.import_module(module_name)
class_ = getattr(module, class_name)
instance = class_()
How to get all of the immediate subdirectories in Python
3 answers
I"m trying to write a simple Python script that will copy a index.tpl to index.html in all of the subdirectories (with a few exceptions).
I"m getting bogged down by trying to get the list of subdirectories.
Answer #1
import os
def get_immediate_subdirectories(a_dir):
return [name for name in os.listdir(a_dir)
if os.path.isdir(os.path.join(a_dir, name))]
How to use multiprocessing queue in Python? sep: Questions
How to print number with commas as thousands separators?
5 answers
I am trying to print an integer in Python 2.6.1 with commas as thousands separators. For example, I want to show the number 1234567 as 1,234,567. How would I go about doing this? I have seen many examples on Google, but I am looking for the simplest practical way.
It does not need to be locale-specific to decide between periods and commas. I would prefer something as simple as reasonably possible.
Answer #1
Locale unaware
"{:,}".format(value) # For Python ‚â•2.7
f"{value:,}" # For Python ‚â•3.6
Locale aware
import locale
locale.setlocale(locale.LC_ALL, "") # Use "" for auto, or force e.g. to "en_US.UTF-8"
"{:n}".format(value) # For Python ‚â•2.7
f"{value:n}" # For Python ‚â•3.6
Reference
Per Format Specification Mini-Language,
The
","option signals the use of a comma for a thousands separator. For a locale aware separator, use the"n"integer presentation type instead.
Answer #2
I got this to work:
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "en_US")
"en_US"
>>> locale.format("%d", 1255000, grouping=True)
"1,255,000"
Sure, you don"t need internationalization support, but it"s clear, concise, and uses a built-in library.
P.S. That "%d" is the usual %-style formatter. You can have only one formatter, but it can be whatever you need in terms of field width and precision settings.
P.P.S. If you can"t get locale to work, I"d suggest a modified version of Mark"s answer:
def intWithCommas(x):
if type(x) not in [type(0), type(0L)]:
raise TypeError("Parameter must be an integer.")
if x < 0:
return "-" + intWithCommas(-x)
result = ""
while x >= 1000:
x, r = divmod(x, 1000)
result = ",%03d%s" % (r, result)
return "%d%s" % (x, result)
Recursion is useful for the negative case, but one recursion per comma seems a bit excessive to me.
How would you make a comma-separated string from a list of strings?
5 answers
What would be your preferred way to concatenate strings from a sequence such that between every two consecutive pairs a comma is added. That is, how do you map, for instance, ["a", "b", "c"] to "a,b,c"? (The cases ["s"] and [] should be mapped to "s" and "", respectively.)
I usually end up using something like "".join(map(lambda x: x+",",l))[:-1], but also feeling somewhat unsatisfied.
Answer #1
my_list = ["a", "b", "c", "d"]
my_string = ",".join(my_list)
"a,b,c,d"
This won"t work if the list contains integers
And if the list contains non-string types (such as integers, floats, bools, None) then do:
my_string = ",".join(map(str, my_list))
