The one of the biggest challenges in the learning code is in course of execution in a bug that seems insoluble. It is almost inevitable that sometimes when code is written you have to seek outside advice .
Often the problem is solved by a better understanding of the programming language you are using and its limitations. In this case, those with experience in that language are usually the best address for Understanding details which are often ignored in lessons.
Aside from this problem, it’s also good to ask questions about the best ways to solve programming problems, and to learn more than one way of doing things from programming professionals . Most of the time, experienced programmers know the fastest and cleanest to do a task.
SoloLearn , despite its name, is all to learn together and get the help you need from professionals. The application is as well a series of courses as it is a social network which allows to communicate with other users with each lesson.
App Preview
SoloLearn was created to be an app. It was released in 2013, but currently has over 35,000,000 users, which is important because SoloLearn is a unique social experience. SoloLearn shines with contests and a helpful community that makes programming feel like a community experience.
The app is designed to teach a lot of different programming languages ‚Äã‚Äãand has courses for HTML, CSS, JavaScript, Java, C++, Python, SQL and many more.
SoloLearn is a free coding app available on iOS App Store, Google Play Store and Desktop via browser. Since there is an app review, we will be focusing on the mobile app instead of the desktop version.
Method of Teaching
Design
SoloLearn features one of the least intuitive interfaces the author has experienced. Without much help, new users entered an interface without a clear first step. Most of the design choices while using the courses and the forum are not intuitive, and the reviewer was often frustrated while using the app.
As the interface becomes easier to use over time, the initial impression is hard to shake. Sometimes the app gives advice on how to use its features, but they seemed to show up later than they should. I came across a lot of these tips in the course of a second lesson, after learning to maneuver the application hard.
In addition, many of the features of this app are not obvious, which adds to the disadvantages. Understanding the comments section, how XP works, and using the power supply was a challenge, and it seemed like the new features just seemed like a fluke. Having a simple tutorial at the start of the app would have improved the experience tremendously.
Features
When it comes to features, SoloLearn has some good qualities and some bad ones. Let’s start with the good ones.
Good
One of the most interesting features of SoloLearn is its robust integrated development environment (IDE). Entering the code isn’t always fun on a floating keyboard, but the powerful built-in IDE autocomplete feature eased much of the pain. Additionally, all code is available on the IDE; When coding with JavaScript, the app has tabs for HTML and CSS, which allows users to access code in these documents.
Another powerful feature is the ability of code written in the export SoloLearn. Although many coding apps are only used for learning, SoloLearn also felt like a free mobile coding editor . In a dedicated space, full code users can publish sophisticated and functional programs such as music players and full user interfaces
Other features include integration with Facebook and Google, a recording tool that allows users to record video of themselves as they code and rank with regional and global ratings.
At no time did the app crash or encounter any serious bugs during the review period, which is always a good sign.
Bad
One of the most notable issues with the application was its load times. Starting a new course took a few minutes for the examiner, with no visible progress bar. This does not happen when starting a new course, so the wait is assumed to result from the course downloading to your device. While this is just a nuisance, having a bar to find out how long the course download might take would make sense.
Also - as mentioned above - many features are hard to understand or find, and some essential features (such as looking in course comments) are nowhere. Tooltips designed to explain features seem to fire haphazardly and rarely when useful.
Cost
Finally, the cost. SoloLearn is advertised as a free code learning application. While there is no price to download the app, SoloLearn is quick to talk about the money. Before even accessing the main content, the app immediately shows an advertisement for their "Pro" plan, which costs $ 6.99 per month or $ 47.99 per year (at time of review). A free trial of the Pro plan is available with an auto-renewing subscription
Without l The Pro subscription, box ads can be found in almost all lessons, and timed interstitial ads (those that cover the screen and require a five-second countdown ) are often activated after filling out forms. Code Coach challenges are not available to non-professional users.
While regular users still ca n’t find new free content every day on SoloLearn, dealing with constant ads and missing out on paid content makes SoloLearn seem less appealing to users who aren’t looking to spend it. ’money.
final thoughts
Although a bit complicated to use, SoloLearn can be a useful learning tool. The 24/7 peer support found in the forum and in the comments is helpful and the amount of content available is incredible. However, this content is packaged in an app with a lackluster user interface and near-constant ads.
Asid and of that, much of the content is aimed at those with some basic programming knowledge . While it is possible to learn programming as a beginner with SoloLearn, the app seems like a better resource for those who know the basics of programming. In this regard, the app is perfect for learning a new programming language, learning an executive, or just brushing up on and completing programming challenges.
SoloLearn Frequently Asked Questions
Javascript Learn Solo: StackOverflow Questions
How do I install pip on macOS or OS X?
I spent most of the day yesterday searching for a clear answer for installing pip (package manager for Python). I can"t find a good solution.
2019: ⚠️easy_install has been deprecated. Check Method #2 below for preferred installation!
Details:
⚡️ OK, I read the solutions given above, but here"s an EASY solution to install pip.
MacOS comes with Python installed. But to make sure that you have Python installed open the terminal and run the following command.
python --version
If this command returns a version number that means Python exists. Which also means that you already have access to easy_install considering you are using macOS/OSX.
ℹ️ Now, all you have to do is run the following command.
sudo easy_install pip
After that, pip will be installed and you"ll be able to use it for installing other packages.
Let me know if you have any problems installing pip this way.
You can install it through Homebrew on OS X. Why would you install Python with Homebrew?
The version of Python that ships with OS X is great for learning but
it’s not good for development. The version shipped with OS X may be
out of date from the official current Python release, which is
considered the stable production version. (source)
Homebrew is something of a package manager for OS X. Find more details on the Homebrew page. Once Homebrew is installed, run the following to install the latest Python, Pip & Setuptools:
brew install python
Answer #4:
I"m surprised no-one has mentioned this - since 2013, python itself is capable of installing pip, no external commands (and no internet connection) required.
sudo -H python -m ensurepip
This will create a similar install to what easy_install would.
Now, you could install external modules. For example
pip install regex # This is only an example for installing other modules
Javascript Learn Solo: StackOverflow Questions
How do I merge two dictionaries in a single expression (taking union of dictionaries)?
Question by Carl Meyer
I have two Python dictionaries, and I want to write a single expression that returns these two dictionaries, merged (i.e. taking the union). The update() method would be what I need, if it returned its result instead of modifying a dictionary in-place.
>>> x = {"a": 1, "b": 2}
>>> y = {"b": 10, "c": 11}
>>> z = x.update(y)
>>> print(z)
None
>>> x
{"a": 1, "b": 10, "c": 11}
How can I get that final merged dictionary in z, not x?
(To be extra-clear, the last-one-wins conflict-handling of dict.update() is what I"m looking for as well.)
Answer #1:
How can I merge two Python dictionaries in a single expression?
For dictionaries x and y, z becomes a shallowly-merged dictionary with values from y replacing those from x.
In Python 3.9.0 or greater (released 17 October 2020): PEP-584, discussed here, was implemented and provides the simplest method:
z = x | y # NOTE: 3.9+ ONLY
In Python 3.5 or greater:
z = {**x, **y}
In Python 2, (or 3.4 or lower) write a function:
def merge_two_dicts(x, y):
z = x.copy() # start with keys and values of x
z.update(y) # modifies z with keys and values of y
return z
and now:
z = merge_two_dicts(x, y)
Explanation
Say you have two dictionaries and you want to merge them into a new dictionary without altering the original dictionaries:
x = {"a": 1, "b": 2}
y = {"b": 3, "c": 4}
The desired result is to get a new dictionary (z) with the values merged, and the second dictionary"s values overwriting those from the first.
However, since many organizations are still on Python 2, you may wish to do this in a backward-compatible way. The classically Pythonic way, available in Python 2 and Python 3.0-3.4, is to do this as a two-step process:
z = x.copy()
z.update(y) # which returns None since it mutates z
In both approaches, y will come second and its values will replace x"s values, thus b will point to 3 in our final result.
Not yet on Python 3.5, but want a single expression
If you are not yet on Python 3.5 or need to write backward-compatible code, and you want this in a single expression, the most performant while the correct approach is to put it in a function:
def merge_two_dicts(x, y):
"""Given two dictionaries, merge them into a new dict as a shallow copy."""
z = x.copy()
z.update(y)
return z
and then you have a single expression:
z = merge_two_dicts(x, y)
You can also make a function to merge an arbitrary number of dictionaries, from zero to a very large number:
def merge_dicts(*dict_args):
"""
Given any number of dictionaries, shallow copy and merge into a new dict,
precedence goes to key-value pairs in latter dictionaries.
"""
result = {}
for dictionary in dict_args:
result.update(dictionary)
return result
This function will work in Python 2 and 3 for all dictionaries. e.g. given dictionaries a to g:
z = merge_dicts(a, b, c, d, e, f, g)
and key-value pairs in g will take precedence over dictionaries a to f, and so on.
Critiques of Other Answers
Don"t use what you see in the formerly accepted answer:
z = dict(x.items() + y.items())
In Python 2, you create two lists in memory for each dict, create a third list in memory with length equal to the length of the first two put together, and then discard all three lists to create the dict. In Python 3, this will fail because you"re adding two dict_items objects together, not two lists -
>>> c = dict(a.items() + b.items())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: "dict_items" and "dict_items"
and you would have to explicitly create them as lists, e.g. z = dict(list(x.items()) + list(y.items())). This is a waste of resources and computation power.
Similarly, taking the union of items() in Python 3 (viewitems() in Python 2.7) will also fail when values are unhashable objects (like lists, for example). Even if your values are hashable, since sets are semantically unordered, the behavior is undefined in regards to precedence. So don"t do this:
>>> c = dict(a.items() | b.items())
This example demonstrates what happens when values are unhashable:
>>> x = {"a": []}
>>> y = {"b": []}
>>> dict(x.items() | y.items())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type: "list"
Here"s an example where y should have precedence, but instead the value from x is retained due to the arbitrary order of sets:
>>> x = {"a": 2}
>>> y = {"a": 1}
>>> dict(x.items() | y.items())
{"a": 2}
Another hack you should not use:
z = dict(x, **y)
This uses the dict constructor and is very fast and memory-efficient (even slightly more so than our two-step process) but unless you know precisely what is happening here (that is, the second dict is being passed as keyword arguments to the dict constructor), it"s difficult to read, it"s not the intended usage, and so it is not Pythonic.
Dictionaries are intended to take hashable keys (e.g. frozensets or tuples), but this method fails in Python 3 when keys are not strings.
>>> c = dict(a, **b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: keyword arguments must be strings
From the mailing list, Guido van Rossum, the creator of the language, wrote:
I am fine with
declaring dict({}, **{1:3}) illegal, since after all it is abuse of
the ** mechanism.
and
Apparently dict(x, **y) is going around as "cool hack" for "call
x.update(y) and return x". Personally, I find it more despicable than
cool.
It is my understanding (as well as the understanding of the creator of the language) that the intended usage for dict(**y) is for creating dictionaries for readability purposes, e.g.:
dict(a=1, b=10, c=11)
instead of
{"a": 1, "b": 10, "c": 11}
Response to comments
Despite what Guido says, dict(x, **y) is in line with the dict specification, which btw. works for both Python 2 and 3. The fact that this only works for string keys is a direct consequence of how keyword parameters work and not a short-coming of dict. Nor is using the ** operator in this place an abuse of the mechanism, in fact, ** was designed precisely to pass dictionaries as keywords.
Again, it doesn"t work for 3 when keys are not strings. The implicit calling contract is that namespaces take ordinary dictionaries, while users must only pass keyword arguments that are strings. All other callables enforced it. dict broke this consistency in Python 2:
>>> foo(**{("a", "b"): None})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() keywords must be strings
>>> dict(**{("a", "b"): None})
{("a", "b"): None}
This inconsistency was bad given other implementations of Python (PyPy, Jython, IronPython). Thus it was fixed in Python 3, as this usage could be a breaking change.
I submit to you that it is malicious incompetence to intentionally write code that only works in one version of a language or that only works given certain arbitrary constraints.
More comments:
dict(x.items() + y.items()) is still the most readable solution for Python 2. Readability counts.
My response: merge_two_dicts(x, y) actually seems much clearer to me, if we"re actually concerned about readability. And it is not forward compatible, as Python 2 is increasingly deprecated.
{**x, **y} does not seem to handle nested dictionaries. the contents of nested keys are simply overwritten, not merged [...] I ended up being burnt by these answers that do not merge recursively and I was surprised no one mentioned it. In my interpretation of the word "merging" these answers describe "updating one dict with another", and not merging.
Yes. I must refer you back to the question, which is asking for a shallow merge of two dictionaries, with the first"s values being overwritten by the second"s - in a single expression.
Assuming two dictionaries of dictionaries, one might recursively merge them in a single function, but you should be careful not to modify the dictionaries from either source, and the surest way to avoid that is to make a copy when assigning values. As keys must be hashable and are usually therefore immutable, it is pointless to copy them:
from copy import deepcopy
def dict_of_dicts_merge(x, y):
z = {}
overlapping_keys = x.keys() & y.keys()
for key in overlapping_keys:
z[key] = dict_of_dicts_merge(x[key], y[key])
for key in x.keys() - overlapping_keys:
z[key] = deepcopy(x[key])
for key in y.keys() - overlapping_keys:
z[key] = deepcopy(y[key])
return z
These approaches are less performant, but they will provide correct behavior.
They will be much less performant than copy and update or the new unpacking because they iterate through each key-value pair at a higher level of abstraction, but they do respect the order of precedence (latter dictionaries have precedence)
You can also chain the dictionaries manually inside a dict comprehension:
{k: v for d in dicts for k, v in d.items()} # iteritems in Python 2.7
or in Python 2.6 (and perhaps as early as 2.4 when generator expressions were introduced):
dict((k, v) for d in dicts for k, v in d.items()) # iteritems in Python 2
itertools.chain will chain the iterators over the key-value pairs in the correct order:
from itertools import chain
z = dict(chain(x.items(), y.items())) # iteritems in Python 2
Performance Analysis
I"m only going to do the performance analysis of the usages known to behave correctly. (Self-contained so you can copy and paste yourself.)
from timeit import repeat
from itertools import chain
x = dict.fromkeys("abcdefg")
y = dict.fromkeys("efghijk")
def merge_two_dicts(x, y):
z = x.copy()
z.update(y)
return z
min(repeat(lambda: {**x, **y}))
min(repeat(lambda: merge_two_dicts(x, y)))
min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
min(repeat(lambda: dict(chain(x.items(), y.items()))))
min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
In Python 3.8.1, NixOS:
>>> min(repeat(lambda: {**x, **y}))
1.0804965235292912
>>> min(repeat(lambda: merge_two_dicts(x, y)))
1.636518670246005
>>> min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
3.1779992282390594
>>> min(repeat(lambda: dict(chain(x.items(), y.items()))))
2.740647904574871
>>> min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
4.266070580109954
$ uname -a
Linux nixos 4.19.113 #1-NixOS SMP Wed Mar 25 07:06:15 UTC 2020 x86_64 GNU/Linux
This will, as you want it, put the final dict in z, and make the value for key b be properly overridden by the second (y) dict"s value:
>>> x = {"a":1, "b": 2}
>>> y = {"b":10, "c": 11}
>>> z = dict(list(x.items()) + list(y.items()))
>>> z
{"a": 1, "c": 11, "b": 10}
If you use Python 2, you can even remove the list() calls. To create z:
>>> z = dict(x.items() + y.items())
>>> z
{"a": 1, "c": 11, "b": 10}
If you use Python version 3.9.0a4 or greater, then you can directly use:
x = {"a":1, "b": 2}
y = {"b":10, "c": 11}
z = x | y
print(z)
{"a": 1, "c": 11, "b": 10}
Answer #3:
An alternative:
z = x.copy()
z.update(y)
Answer #4:
Another, more concise, option:
z = dict(x, **y)
Note: this has become a popular answer, but it is important to point out that if y has any non-string keys, the fact that this works at all is an abuse of a CPython implementation detail, and it does not work in Python 3, or in PyPy, IronPython, or Jython. Also, Guido is not a fan. So I can"t recommend this technique for forward-compatible or cross-implementation portable code, which really means it should be avoided entirely.
Answer #5:
This probably won"t be a popular answer, but you almost certainly do not want to do this. If you want a copy that"s a merge, then use copy (or deepcopy, depending on what you want) and then update. The two lines of code are much more readable - more Pythonic - than the single line creation with .items() + .items(). Explicit is better than implicit.
In addition, when you use .items() (pre Python 3.0), you"re creating a new list that contains the items from the dict. If your dictionaries are large, then that is quite a lot of overhead (two large lists that will be thrown away as soon as the merged dict is created). update() can work more efficiently, because it can run through the second dict item-by-item.
IMO the tiny slowdown between the first two is worth it for the readability. In addition, keyword arguments for dictionary creation was only added in Python 2.3, whereas copy() and update() will work in older versions.
Javascript Learn Solo: StackOverflow Questions
Finding the index of an item in a list
Given a list ["foo", "bar", "baz"] and an item in the list "bar", how do I get its index (1) in Python?
Note that while this is perhaps the cleanest way to answer the question as asked, index is a rather weak component of the list API, and I can"t remember the last time I used it in anger. It"s been pointed out to me in the comments that because this answer is heavily referenced, it should be made more complete. Some caveats about list.index follow. It is probably worth initially taking a look at the documentation for it:
list.index(x[, start[, end]])
Return zero-based index in the list of the first item whose value is equal to x. Raises a ValueError if there is no such item.
The optional arguments start and end are interpreted as in the slice notation and are used to limit the search to a particular subsequence of the list. The returned index is computed relative to the beginning of the full sequence rather than the start argument.
Linear time-complexity in list length
An index call checks every element of the list in order, until it finds a match. If your list is long, and you don"t know roughly where in the list it occurs, this search could become a bottleneck. In that case, you should consider a different data structure. Note that if you know roughly where to find the match, you can give index a hint. For instance, in this snippet, l.index(999_999, 999_990, 1_000_000) is roughly five orders of magnitude faster than straight l.index(999_999), because the former only has to search 10 entries, while the latter searches a million:
Only returns the index of the first match to its argument
A call to index searches through the list in order until it finds a match, and stops there. If you expect to need indices of more matches, you should use a list comprehension, or generator expression.
>>> [1, 1].index(1)
0
>>> [i for i, e in enumerate([1, 2, 1]) if e == 1]
[0, 2]
>>> g = (i for i, e in enumerate([1, 2, 1]) if e == 1)
>>> next(g)
0
>>> next(g)
2
Most places where I once would have used index, I now use a list comprehension or generator expression because they"re more generalizable. So if you"re considering reaching for index, take a look at these excellent Python features.
Throws if element not present in list
A call to index results in a ValueError if the item"s not present.
>>> [1, 1].index(2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: 2 is not in list
If the item might not be present in the list, you should either
Check for it first with item in my_list (clean, readable approach), or
Wrap the index call in a try/except block which catches ValueError (probably faster, at least when the list to search is long, and the item is usually present.)
Answer #2:
One thing that is really helpful in learning Python is to use the interactive help function:
>>> help(["foo", "bar", "baz"])
Help on list object:
class list(object)
...
|
| index(...)
| L.index(value, [start, [stop]]) -> integer -- return first index of value
|
which will often lead you to the method you are looking for.
Answer #3:
The majority of answers explain how to find a single index, but their methods do not return multiple indexes if the item is in the list multiple times. Use enumerate():
for i, j in enumerate(["foo", "bar", "baz"]):
if j == "bar":
print(i)
The index() function only returns the first occurrence, while enumerate() returns all occurrences.
As a list comprehension:
[i for i, j in enumerate(["foo", "bar", "baz"]) if j == "bar"]
Here"s also another small solution with itertools.count() (which is pretty much the same approach as enumerate):
from itertools import izip as zip, count # izip for maximum efficiency
[i for i, j in zip(count(), ["foo", "bar", "baz"]) if j == "bar"]
This is more efficient for larger lists than using enumerate():
$ python -m timeit -s "from itertools import izip as zip, count" "[i for i, j in zip(count(), ["foo", "bar", "baz"]*500) if j == "bar"]"
10000 loops, best of 3: 174 usec per loop
$ python -m timeit "[i for i, j in enumerate(["foo", "bar", "baz"]*500) if j == "bar"]"
10000 loops, best of 3: 196 usec per loop
Answer #4:
To get all indexes:
indexes = [i for i,x in enumerate(xs) if x == "foo"]
Answer #5:
index() returns the first index of value!
| index(...)
| L.index(value, [start, [stop]]) -> integer -- return first index of value
def all_indices(value, qlist):
indices = []
idx = -1
while True:
try:
idx = qlist.index(value, idx+1)
indices.append(idx)
except ValueError:
break
return indices
all_indices("foo", ["foo";"bar";"baz";"foo"])
Javascript Learn Solo: StackOverflow Questions
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately
Tried to perform REST GET through python requests with the following code and I got error.
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:79:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail.
For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
My python version is 2.7.3. I tried to install urllib3 and requests[security] as some other thread suggests, I still got the same error.
Dynamic instantiation from string name of a class in dynamically imported module?
In python, I have to instantiate certain class, knowing its name in a string, but this class "lives" in a dynamically imported module. An example follows:
loader-class script:
import sys
class loader:
def __init__(self, module_name, class_name): # both args are strings
try:
__import__(module_name)
modul = sys.modules[module_name]
instance = modul.class_name() # obviously this doesn"t works, here is my main problem!
except ImportError:
# manage import error
some-dynamically-loaded-module script:
class myName:
# etc...
I use this arrangement to make any dynamically-loaded-module to be used by the loader-class following certain predefined behaviours in the dyn-loaded-modules...
Import the root module with importlib.import_module and load the class by its name using getattr function:
# Standard import
import importlib
# Load "module.submodule.MyClass"
MyClass = getattr(importlib.import_module("module.submodule"), "MyClass")
# Instantiate the class (pass arguments to the constructor, if needed)
instance = MyClass()
explanations
You probably don"t want to use __import__ to dynamically import a module by name, as it does not allow you to import submodules:
>>> mod = __import__("os.path")
>>> mod.join
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: "module" object has no attribute "join"
Here is what the python doc says about __import__:
Note: This is an advanced function that is not needed in everyday
Python programming, unlike importlib.import_module().
Instead, use the standard importlib module to dynamically import a module by name. With getattr you can then instantiate a class by its name:
This code is valid in python ‚â• 2.7 (including python 3).
pandas loc vs. iloc vs. at vs. iat?
Recently began branching out from my safe place (R) into Python and and am a bit confused by the cell localization/selection in Pandas. I"ve read the documentation but I"m struggling to understand the practical implications of the various localization/selection options.
Is there a reason why I should ever use .loc or .iloc over at, and iat or vice versa? In what situations should I use which method?
Note: future readers be aware that this question is old and was written before pandas v0.20 when there used to exist a function called .ix. This method was later split into two - loc and iloc - to make the explicit distinction between positional and label based indexing. Please beware that ix was discontinued due to inconsistent behavior and being hard to grok, and no longer exists in current versions of pandas (>= 1.0).
Answer #1:
loc: only work on index iloc: work on position at: get scalar values. It"s a very fast loc iat: Get scalar values. It"s a very fast iloc
Also,
at and iat are meant to access a scalar, that is, a single element
in the dataframe, while loc and iloc are ments to access several
elements at the same time, potentially to perform vectorized
operations.
I want to send a datetime.datetime object in serialized form from Python using JSON and de-serialize in JavaScript using JSON. What is the best way to do this?
Answer #1:
You can add the "default" parameter to json.dumps to handle this:
def handler(obj):
if hasattr(obj, "isoformat"):
return obj.isoformat()
elif isinstance(obj, ...):
return ...
else:
raise TypeError, "Object of type %s with value of %s is not JSON serializable" % (type(obj), repr(obj))
Update: Added output of type as well as value.
Update: Also handle date
Javascript equivalent of Python"s zip function
Is there a javascript equivalent of Python"s zip function? That is, given multiple arrays of equal lengths create an array of pairs.
For instance, if I have three arrays that look like this:
var array1 = [1, 2, 3];
var array2 = ["a","b","c"];
var array3 = [4, 5, 6];
The output array should be:
var output array:[[1,"a",4], [2,"b",5], [3,"c",6]]
(and FizzyTea points out that ES6 has variadic argument syntax, so the following function definition will act like python, but see below for disclaimer... this will not be its own inverse so zip(zip(x)) will not equal x; though as Matt Kramer points out zip(...zip(...x))==x (like in regular python zip(*zip(*x))==x))
Alternative definition equiv. to Python{zip}:
> zip = (...rows) => [...rows[0]].map((_,c) => rows.map(row => row[c]))
> zip( ["row0col0", "row0col1", "row0col2"] ,
["row1col0", "row1col1", "row1col2"] );
// note zip(row0,row1), not zip(matrix)
same answer as above
(Do note that the ... syntax may have performance issues at this time, and possibly in the future, so if you use the second answer with variadic arguments, you may want to perf test it. That said it"s been quite a while since it"s been in the standard.)
Make sure to note the addendum if you wish to use this on strings (perhaps there"s a better way to do it now with es6 iterables).
Here"s a oneliner:
function zip(arrays) {
return arrays[0].map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
// > zip([[1,2],[11,22],[111,222]])
// [[1,11,111],[2,22,222]]]
// If you believe the following is a valid return value:
// > zip([])
// []
// then you can special-case it, or just do
// return arrays.length==0 ? [] : arrays[0].map(...)
The above assumes that the arrays are of equal size, as they should be. It also assumes you pass in a single list of lists argument, unlike Python"s version where the argument list is variadic. If you want all of these "features", see below. It takes just about 2 extra lines of code.
The following will mimic Python"s zip behavior on edge cases where the arrays are not of equal size, silently pretending the longer parts of arrays don"t exist:
function zip() {
var args = [].slice.call(arguments);
var shortest = args.length==0 ? [] : args.reduce(function(a,b){
return a.length<b.length ? a : b
});
return shortest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222]]]
// > zip()
// []
This will mimic Python"s itertools.zip_longest behavior, inserting undefined where arrays are not defined:
function zip() {
var args = [].slice.call(arguments);
var longest = args.reduce(function(a,b){
return a.length>b.length ? a : b
}, []);
return longest.map(function(_,i){
return args.map(function(array){return array[i]})
});
}
// > zip([1,2],[11,22],[111,222,333])
// [[1,11,111],[2,22,222],[null,null,333]]
// > zip()
// []
If you use these last two version (variadic aka. multiple-argument versions), then zip is no longer its own inverse. To mimic the zip(*[...]) idiom from Python, you will need to do zip.apply(this, [...]) when you want to invert the zip function or if you want to similarly have a variable number of lists as input.
addendum:
To make this handle any iterable (e.g. in Python you can use zip on strings, ranges, map objects, etc.), you could define the following:
function iterView(iterable) {
// returns an array equivalent to the iterable
}
However if you write zip in the following way, even that won"t be necessary:
function zip(arrays) {
return Array.apply(null,Array(arrays[0].length)).map(function(_,i){
return arrays.map(function(array){return array[i]})
});
}
(Or you could use a range(...) Python-style function if you"ve written one already. Eventually you will be able to use ECMAScript array comprehensions or generators.)
What blocks Ruby, Python to get Javascript V8 speed?
Are there any Ruby / Python features that are blocking implementation of optimizations (e.g. inline caching) V8 engine has?
Python is co-developed by Google guys so it shouldn"t be blocked by software patents.
Or this is rather matter of resources put into the V8 project by Google.
Answer #1:
What blocks Ruby, Python to get Javascript V8 speed?
Nothing.
Well, okay: money. (And time, people, resources, but if you have money, you can buy those.)
V8 has a team of brilliant, highly-specialized, highly-experienced (and thus highly-paid) engineers working on it, that have decades of experience (I"m talking individually – collectively it"s more like centuries) in creating high-performance execution engines for dynamic OO languages. They are basically the same people who also created the Sun HotSpot JVM (among many others).
Lars Bak, the lead developer, has been literally working on VMs for 25 years (and all of those VMs have lead up to V8), which is basically his entire (professional) life. Some of the people writing Ruby VMs aren"t even 25 years old.
Are there any Ruby / Python features that are blocking implementation of optimizations (e.g. inline caching) V8 engine has?
Given that at least IronRuby, JRuby, MagLev, MacRuby and Rubinius have either monomorphic (IronRuby) or polymorphic inline caching, the answer is obviously no.
Modern Ruby implementations already do a great deal of optimizations. For example, for certain operations, Rubinius"s Hash class is faster than YARV"s. Now, this doesn"t sound terribly exciting until you realize that Rubinius"s Hash class is implemented in 100% pure Ruby, while YARV"s is implemented in 100% hand-optimized C.
So, at least in some cases, Rubinius can generate better code than GCC!
Or this is rather matter of resources put into the V8 project by Google.
Yes. Not just Google. The lineage of V8"s source code is 25 years old now. The people who are working on V8 also created the Self VM (to this day one of the fastest dynamic OO language execution engines ever created), the Animorphic Smalltalk VM (to this day one of the fastest Smalltalk execution engines ever created), the HotSpot JVM (the fastest JVM ever created, probably the fastest VM period) and OOVM (one of the most efficient Smalltalk VMs ever created).
In fact, Lars Bak, the lead developer of V8, worked on every single one of those, plus a few others.
Django Template Variables and Javascript
When I render a page using the Django template renderer, I can pass in a dictionary variable containing various values to manipulate them in the page using {{ myVar }}.
Is there a way to access the same variable in Javascript (perhaps using the DOM, I don"t know how Django makes the variables accessible)? I want to be able to lookup details using an AJAX lookup based on the values contained in the variables passed in.
Answer #1:
The {{variable}} is substituted directly into the HTML. Do a view source; it isn"t a "variable" or anything like it. It"s just rendered text.
Having said that, you can put this kind of substitution into your JavaScript.
<script type="text/javascript">
var a = "{{someDjangoVariable}}";
</script>
This gives you "dynamic" javascript.
Web-scraping JavaScript page with Python
I"m trying to develop a simple web scraper. I want to extract text without the HTML code. In fact, I achieve this goal, but I have seen that in some pages where JavaScript is loaded I didn"t obtain good results.
For example, if some JavaScript code adds some text, I can"t see it, because when I call
response = urllib2.urlopen(request)
I get the original text without the added one (because JavaScript is executed in the client).
So, I"m looking for some ideas to solve this problem.
Answer #1:
EDIT 30/Dec/2017: This answer appears in top results of Google searches, so I decided to update it. The old answer is still at the end.
dryscape isn"t maintained anymore and the library dryscape developers recommend is Python 2 only. I have found using Selenium"s python library with Phantom JS as a web driver fast enough and easy to get the work done.
Once you have installed Phantom JS, make sure the phantomjs binary is available in the current path:
phantomjs --version
# result:
2.1.1
Example
To give an example, I created a sample page with following HTML code. (link):
Or if you already have a datetime object that is TZ aware (not naive):
# This timestamp is in UTC
my_ct = datetime.datetime.now(tz=pytz.UTC)
# Now convert it to another timezone
new_ct = my_ct.astimezone(tz)
>>> new_ct.isoformat()
2017-01-13T11:29:22.601991-05:00
Python strptime() and timezones?
I have a CSV dumpfile from a Blackberry IPD backup, created using IPDDump.
The date/time strings in here look something like this
(where EST is an Australian time-zone):
Tue Jun 22 07:46:22 EST 2010
I need to be able to parse this date in Python. At first, I tried to use the strptime() function from datettime.
However, for some reason, the datetime object that comes back doesn"t seem to have any tzinfo associated with it.
I did read on this page that apparently datetime.strptime silently discards tzinfo, however, I checked the documentation, and I can"t find anything to that effect documented here.
I have been able to get the date parsed using a third-party Python library, dateutil, however I"m still curious as to how I was using the in-built strptime() incorrectly? Is there any way to get strptime() to play nicely with timezones?
Answer #1:
I recommend using python-dateutil. Its parser has been able to parse every date format I"ve thrown at it so far.
and so on. No dealing with strptime() format nonsense... just throw a date at it and it Does The Right Thing.
Update: Oops. I missed in your original question that you mentioned that you used dateutil, sorry about that. But I hope this answer is still useful to other people who stumble across this question when they have date parsing questions and see the utility of that module.
Fitting empirical distribution to theoretical ones with Scipy (Python)?
INTRODUCTION: I have a list of more than 30,000 integer values ranging from 0 to 47, inclusive, e.g.[0,0,0,0,..,1,1,1,1,...,2,2,2,2,...,47,47,47,...] sampled from some continuous distribution. The values in the list are not necessarily in order, but order doesn"t matter for this problem.
PROBLEM: Based on my distribution I would like to calculate p-value (the probability of seeing greater values) for any given value. For example, as you can see p-value for 0 would be approaching 1 and p-value for higher numbers would be tending to 0.
I don"t know if I am right, but to determine probabilities I think I need to fit my data to a theoretical distribution that is the most suitable to describe my data. I assume that some kind of goodness of fit test is needed to determine the best model.
Is there a way to implement such an analysis in Python (Scipy or Numpy)?
Could you present any examples?
Thank you!
Answer #1:
Distribution Fitting with Sum of Square Error (SSE)
This is an update and modification to Saullo"s answer, that uses the full list of the current scipy.stats distributions and returns the distribution with the least SSE between the distribution"s histogram and the data"s histogram.
Example Fitting
Using the El Niño dataset from statsmodels, the distributions are fit and error is determined. The distribution with the least error is returned.
All Distributions
Best Fit Distribution
Example Code
%matplotlib inline
import warnings
import numpy as np
import pandas as pd
import scipy.stats as st
import statsmodels.api as sm
from scipy.stats._continuous_distns import _distn_names
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams["figure.figsize"] = (16.0, 12.0)
matplotlib.style.use("ggplot")
# Create models from data
def best_fit_distribution(data, bins=200, ax=None):
"""Model data by finding best fit distribution to data"""
# Get histogram of original data
y, x = np.histogram(data, bins=bins, density=True)
x = (x + np.roll(x, -1))[:-1] / 2.0
# Best holders
best_distributions = []
# Estimate distribution parameters from data
for ii, distribution in enumerate([d for d in _distn_names if not d in ["levy_stable", "studentized_range"]]):
print("{:>3} / {:<3}: {}".format( ii+1, len(_distn_names), distribution ))
distribution = getattr(st, distribution)
# Try to fit the distribution
try:
# Ignore warnings from data that can"t be fit
with warnings.catch_warnings():
warnings.filterwarnings("ignore")
# fit dist to data
params = distribution.fit(data)
# Separate parts of parameters
arg = params[:-2]
loc = params[-2]
scale = params[-1]
# Calculate fitted PDF and error with fit in distribution
pdf = distribution.pdf(x, loc=loc, scale=scale, *arg)
sse = np.sum(np.power(y - pdf, 2.0))
# if axis pass in add to plot
try:
if ax:
pd.Series(pdf, x).plot(ax=ax)
end
except Exception:
pass
# identify if this distribution is better
best_distributions.append((distribution, params, sse))
except Exception:
pass
return sorted(best_distributions, key=lambda x:x[2])
def make_pdf(dist, params, size=10000):
"""Generate distributions"s Probability Distribution Function """
# Separate parts of parameters
arg = params[:-2]
loc = params[-2]
scale = params[-1]
# Get sane start and end points of distribution
start = dist.ppf(0.01, *arg, loc=loc, scale=scale) if arg else dist.ppf(0.01, loc=loc, scale=scale)
end = dist.ppf(0.99, *arg, loc=loc, scale=scale) if arg else dist.ppf(0.99, loc=loc, scale=scale)
# Build PDF and turn into pandas Series
x = np.linspace(start, end, size)
y = dist.pdf(x, loc=loc, scale=scale, *arg)
pdf = pd.Series(y, x)
return pdf
# Load data from statsmodels datasets
data = pd.Series(sm.datasets.elnino.load_pandas().data.set_index("YEAR").values.ravel())
# Plot for comparison
plt.figure(figsize=(12,8))
ax = data.plot(kind="hist", bins=50, density=True, alpha=0.5, color=list(matplotlib.rcParams["axes.prop_cycle"])[1]["color"])
# Save plot limits
dataYLim = ax.get_ylim()
# Find best fit distribution
best_distibutions = best_fit_distribution(data, 200, ax)
best_dist = best_distibutions[0]
# Update plots
ax.set_ylim(dataYLim)
ax.set_title(u"El Niño sea temp.
All Fitted Distributions")
ax.set_xlabel(u"Temp (°C)")
ax.set_ylabel("Frequency")
# Make PDF with best params
pdf = make_pdf(best_dist[0], best_dist[1])
# Display
plt.figure(figsize=(12,8))
ax = pdf.plot(lw=2, label="PDF", legend=True)
data.plot(kind="hist", bins=50, density=True, alpha=0.5, label="Data", legend=True, ax=ax)
param_names = (best_dist[0].shapes + ", loc, scale").split(", ") if best_dist[0].shapes else ["loc", "scale"]
param_str = ", ".join(["{}={:0.2f}".format(k,v) for k,v in zip(param_names, best_dist[1])])
dist_str = "{}({})".format(best_dist[0].name, param_str)
ax.set_title(u"El Niño sea temp. with best fit distribution
" + dist_str)
ax.set_xlabel(u"Temp. (°C)")
ax.set_ylabel("Frequency")
How can I open multiple files using "with open" in Python?
I want to change a couple of files at one time, iff I can write to all of them. I"m wondering if I somehow can combine the multiple open calls with the with statement:
try:
with open("a", "w") as a and open("b", "w") as b:
do_something()
except IOError as e:
print "Operation failed: %s" % e.strerror
If that"s not possible, what would an elegant solution to this problem look like?
Answer #1:
As of Python 2.7 (or 3.1 respectively) you can write
with open("a", "w") as a, open("b", "w") as b:
do_something()
In earlier versions of Python, you can sometimes use
contextlib.nested() to nest context managers. This won"t work as expected for opening multiples files, though -- see the linked documentation for details.
In the rare case that you want to open a variable number of files all at the same time, you can use contextlib.ExitStack, starting from Python version 3.3:
with ExitStack() as stack:
files = [stack.enter_context(open(fname)) for fname in filenames]
# Do something with "files"
Most of the time you have a variable set of files, you likely want to open them one after the other, though.
Answer #2:
For opening many files at once or for long file paths, it may be useful to break things up over multiple lines. From the Python Style Guide as suggested by @Sven Marnach in comments to another answer:
with open("/path/to/InFile.ext", "r") as file_1,
open("/path/to/OutFile.ext", "w") as file_2:
file_2.write(file_1.read())
open() in Python does not create a file if it doesn"t exist
What is the best way to open a file as read/write if it exists, or if it does not, then create it and open it as read/write? From what I read, file = open("myfile.dat", "rw") should do this, right?
It is not working for me (Python 2.6.2) and I"m wondering if it is a version problem, or not supposed to work like that or what.
The bottom line is, I just need a solution for the problem. I am curious about the other stuff, but all I need is a nice way to do the opening part.
The enclosing directory was writeable by user and group, not other (I"m on a Linux system... so permissions 775 in other words), and the exact error was:
IOError: no such file or directory.
Answer #1:
You should use open with the w+ mode:
file = open("myfile.dat", "w+")
Answer #2:
The advantage of the following approach is that the file is properly closed at the block"s end, even if an exception is raised on the way. It"s equivalent to try-finally, but much shorter.
with open("file.dat";"a+") as f:
f.write(...)
...
a+ Opens a file for both appending and reading. The file pointer is
at the end of the file if the file exists. The file opens in the
append mode. If the file does not exist, it creates a new file for
reading and writing. -Python file modes
f.seek(pos [, (0|1|2)])
pos .. position of the r/w pointer
[] .. optionally
() .. one of ->
0 .. absolute position
1 .. relative position to current
2 .. relative position from end
Only "rwab+" characters are allowed; there must be exactly one of "rwa" - see Stack Overflow question Python file modes detail.
Difference between modes a, a+, w, w+, and r+ in built-in open function?
In the python built-in open function, what is the exact difference between the modes w, a, w+, a+, and r+?
In particular, the documentation implies that all of these will allow writing to the file, and says that it opens the files for "appending", "writing", and "updating" specifically, but does not define what these terms mean.
Answer #1:
The opening modes are exactly the same as those for the C standard library function fopen().
The argument mode points to a string beginning with one of the following
sequences (Additional characters may follow these sequences.):
``r"" Open text file for reading. The stream is positioned at the
beginning of the file.
``r+"" Open for reading and writing. The stream is positioned at the
beginning of the file.
``w"" Truncate file to zero length or create text file for writing.
The stream is positioned at the beginning of the file.
``w+"" Open for reading and writing. The file is created if it does not
exist, otherwise it is truncated. The stream is positioned at
the beginning of the file.
``a"" Open for writing. The file is created if it does not exist. The
stream is positioned at the end of the file. Subsequent writes
to the file will always end up at the then current end of file,
irrespective of any intervening fseek(3) or similar.
``a+"" Open for reading and writing. The file is created if it does not
exist. The stream is positioned at the end of the file. Subse-
quent writes to the file will always end up at the then current
end of file, irrespective of any intervening fseek(3) or similar.
If you don"t like that empty tuple at the beginning, you can just change the range statement to range(1, len(s)+1) to avoid a 0-length combination.
Javascript Learn Solo: StackOverflow Questions
How do I merge two dictionaries in a single expression (taking union of dictionaries)?
Question by Carl Meyer
I have two Python dictionaries, and I want to write a single expression that returns these two dictionaries, merged (i.e. taking the union). The update() method would be what I need, if it returned its result instead of modifying a dictionary in-place.
>>> x = {"a": 1, "b": 2}
>>> y = {"b": 10, "c": 11}
>>> z = x.update(y)
>>> print(z)
None
>>> x
{"a": 1, "b": 10, "c": 11}
How can I get that final merged dictionary in z, not x?
(To be extra-clear, the last-one-wins conflict-handling of dict.update() is what I"m looking for as well.)
Answer #1:
How can I merge two Python dictionaries in a single expression?
For dictionaries x and y, z becomes a shallowly-merged dictionary with values from y replacing those from x.
In Python 3.9.0 or greater (released 17 October 2020): PEP-584, discussed here, was implemented and provides the simplest method:
z = x | y # NOTE: 3.9+ ONLY
In Python 3.5 or greater:
z = {**x, **y}
In Python 2, (or 3.4 or lower) write a function:
def merge_two_dicts(x, y):
z = x.copy() # start with keys and values of x
z.update(y) # modifies z with keys and values of y
return z
and now:
z = merge_two_dicts(x, y)
Explanation
Say you have two dictionaries and you want to merge them into a new dictionary without altering the original dictionaries:
x = {"a": 1, "b": 2}
y = {"b": 3, "c": 4}
The desired result is to get a new dictionary (z) with the values merged, and the second dictionary"s values overwriting those from the first.
However, since many organizations are still on Python 2, you may wish to do this in a backward-compatible way. The classically Pythonic way, available in Python 2 and Python 3.0-3.4, is to do this as a two-step process:
z = x.copy()
z.update(y) # which returns None since it mutates z
In both approaches, y will come second and its values will replace x"s values, thus b will point to 3 in our final result.
Not yet on Python 3.5, but want a single expression
If you are not yet on Python 3.5 or need to write backward-compatible code, and you want this in a single expression, the most performant while the correct approach is to put it in a function:
def merge_two_dicts(x, y):
"""Given two dictionaries, merge them into a new dict as a shallow copy."""
z = x.copy()
z.update(y)
return z
and then you have a single expression:
z = merge_two_dicts(x, y)
You can also make a function to merge an arbitrary number of dictionaries, from zero to a very large number:
def merge_dicts(*dict_args):
"""
Given any number of dictionaries, shallow copy and merge into a new dict,
precedence goes to key-value pairs in latter dictionaries.
"""
result = {}
for dictionary in dict_args:
result.update(dictionary)
return result
This function will work in Python 2 and 3 for all dictionaries. e.g. given dictionaries a to g:
z = merge_dicts(a, b, c, d, e, f, g)
and key-value pairs in g will take precedence over dictionaries a to f, and so on.
Critiques of Other Answers
Don"t use what you see in the formerly accepted answer:
z = dict(x.items() + y.items())
In Python 2, you create two lists in memory for each dict, create a third list in memory with length equal to the length of the first two put together, and then discard all three lists to create the dict. In Python 3, this will fail because you"re adding two dict_items objects together, not two lists -
>>> c = dict(a.items() + b.items())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: "dict_items" and "dict_items"
and you would have to explicitly create them as lists, e.g. z = dict(list(x.items()) + list(y.items())). This is a waste of resources and computation power.
Similarly, taking the union of items() in Python 3 (viewitems() in Python 2.7) will also fail when values are unhashable objects (like lists, for example). Even if your values are hashable, since sets are semantically unordered, the behavior is undefined in regards to precedence. So don"t do this:
>>> c = dict(a.items() | b.items())
This example demonstrates what happens when values are unhashable:
>>> x = {"a": []}
>>> y = {"b": []}
>>> dict(x.items() | y.items())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type: "list"
Here"s an example where y should have precedence, but instead the value from x is retained due to the arbitrary order of sets:
>>> x = {"a": 2}
>>> y = {"a": 1}
>>> dict(x.items() | y.items())
{"a": 2}
Another hack you should not use:
z = dict(x, **y)
This uses the dict constructor and is very fast and memory-efficient (even slightly more so than our two-step process) but unless you know precisely what is happening here (that is, the second dict is being passed as keyword arguments to the dict constructor), it"s difficult to read, it"s not the intended usage, and so it is not Pythonic.
Dictionaries are intended to take hashable keys (e.g. frozensets or tuples), but this method fails in Python 3 when keys are not strings.
>>> c = dict(a, **b)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: keyword arguments must be strings
From the mailing list, Guido van Rossum, the creator of the language, wrote:
I am fine with
declaring dict({}, **{1:3}) illegal, since after all it is abuse of
the ** mechanism.
and
Apparently dict(x, **y) is going around as "cool hack" for "call
x.update(y) and return x". Personally, I find it more despicable than
cool.
It is my understanding (as well as the understanding of the creator of the language) that the intended usage for dict(**y) is for creating dictionaries for readability purposes, e.g.:
dict(a=1, b=10, c=11)
instead of
{"a": 1, "b": 10, "c": 11}
Response to comments
Despite what Guido says, dict(x, **y) is in line with the dict specification, which btw. works for both Python 2 and 3. The fact that this only works for string keys is a direct consequence of how keyword parameters work and not a short-coming of dict. Nor is using the ** operator in this place an abuse of the mechanism, in fact, ** was designed precisely to pass dictionaries as keywords.
Again, it doesn"t work for 3 when keys are not strings. The implicit calling contract is that namespaces take ordinary dictionaries, while users must only pass keyword arguments that are strings. All other callables enforced it. dict broke this consistency in Python 2:
>>> foo(**{("a", "b"): None})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() keywords must be strings
>>> dict(**{("a", "b"): None})
{("a", "b"): None}
This inconsistency was bad given other implementations of Python (PyPy, Jython, IronPython). Thus it was fixed in Python 3, as this usage could be a breaking change.
I submit to you that it is malicious incompetence to intentionally write code that only works in one version of a language or that only works given certain arbitrary constraints.
More comments:
dict(x.items() + y.items()) is still the most readable solution for Python 2. Readability counts.
My response: merge_two_dicts(x, y) actually seems much clearer to me, if we"re actually concerned about readability. And it is not forward compatible, as Python 2 is increasingly deprecated.
{**x, **y} does not seem to handle nested dictionaries. the contents of nested keys are simply overwritten, not merged [...] I ended up being burnt by these answers that do not merge recursively and I was surprised no one mentioned it. In my interpretation of the word "merging" these answers describe "updating one dict with another", and not merging.
Yes. I must refer you back to the question, which is asking for a shallow merge of two dictionaries, with the first"s values being overwritten by the second"s - in a single expression.
Assuming two dictionaries of dictionaries, one might recursively merge them in a single function, but you should be careful not to modify the dictionaries from either source, and the surest way to avoid that is to make a copy when assigning values. As keys must be hashable and are usually therefore immutable, it is pointless to copy them:
from copy import deepcopy
def dict_of_dicts_merge(x, y):
z = {}
overlapping_keys = x.keys() & y.keys()
for key in overlapping_keys:
z[key] = dict_of_dicts_merge(x[key], y[key])
for key in x.keys() - overlapping_keys:
z[key] = deepcopy(x[key])
for key in y.keys() - overlapping_keys:
z[key] = deepcopy(y[key])
return z
These approaches are less performant, but they will provide correct behavior.
They will be much less performant than copy and update or the new unpacking because they iterate through each key-value pair at a higher level of abstraction, but they do respect the order of precedence (latter dictionaries have precedence)
You can also chain the dictionaries manually inside a dict comprehension:
{k: v for d in dicts for k, v in d.items()} # iteritems in Python 2.7
or in Python 2.6 (and perhaps as early as 2.4 when generator expressions were introduced):
dict((k, v) for d in dicts for k, v in d.items()) # iteritems in Python 2
itertools.chain will chain the iterators over the key-value pairs in the correct order:
from itertools import chain
z = dict(chain(x.items(), y.items())) # iteritems in Python 2
Performance Analysis
I"m only going to do the performance analysis of the usages known to behave correctly. (Self-contained so you can copy and paste yourself.)
from timeit import repeat
from itertools import chain
x = dict.fromkeys("abcdefg")
y = dict.fromkeys("efghijk")
def merge_two_dicts(x, y):
z = x.copy()
z.update(y)
return z
min(repeat(lambda: {**x, **y}))
min(repeat(lambda: merge_two_dicts(x, y)))
min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
min(repeat(lambda: dict(chain(x.items(), y.items()))))
min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
In Python 3.8.1, NixOS:
>>> min(repeat(lambda: {**x, **y}))
1.0804965235292912
>>> min(repeat(lambda: merge_two_dicts(x, y)))
1.636518670246005
>>> min(repeat(lambda: {k: v for d in (x, y) for k, v in d.items()}))
3.1779992282390594
>>> min(repeat(lambda: dict(chain(x.items(), y.items()))))
2.740647904574871
>>> min(repeat(lambda: dict(item for d in (x, y) for item in d.items())))
4.266070580109954
$ uname -a
Linux nixos 4.19.113 #1-NixOS SMP Wed Mar 25 07:06:15 UTC 2020 x86_64 GNU/Linux
This will, as you want it, put the final dict in z, and make the value for key b be properly overridden by the second (y) dict"s value:
>>> x = {"a":1, "b": 2}
>>> y = {"b":10, "c": 11}
>>> z = dict(list(x.items()) + list(y.items()))
>>> z
{"a": 1, "c": 11, "b": 10}
If you use Python 2, you can even remove the list() calls. To create z:
>>> z = dict(x.items() + y.items())
>>> z
{"a": 1, "c": 11, "b": 10}
If you use Python version 3.9.0a4 or greater, then you can directly use:
x = {"a":1, "b": 2}
y = {"b":10, "c": 11}
z = x | y
print(z)
{"a": 1, "c": 11, "b": 10}
Answer #3:
An alternative:
z = x.copy()
z.update(y)
Answer #4:
Another, more concise, option:
z = dict(x, **y)
Note: this has become a popular answer, but it is important to point out that if y has any non-string keys, the fact that this works at all is an abuse of a CPython implementation detail, and it does not work in Python 3, or in PyPy, IronPython, or Jython. Also, Guido is not a fan. So I can"t recommend this technique for forward-compatible or cross-implementation portable code, which really means it should be avoided entirely.
Answer #5:
This probably won"t be a popular answer, but you almost certainly do not want to do this. If you want a copy that"s a merge, then use copy (or deepcopy, depending on what you want) and then update. The two lines of code are much more readable - more Pythonic - than the single line creation with .items() + .items(). Explicit is better than implicit.
In addition, when you use .items() (pre Python 3.0), you"re creating a new list that contains the items from the dict. If your dictionaries are large, then that is quite a lot of overhead (two large lists that will be thrown away as soon as the merged dict is created). update() can work more efficiently, because it can run through the second dict item-by-item.
IMO the tiny slowdown between the first two is worth it for the readability. In addition, keyword arguments for dictionary creation was only added in Python 2.3, whereas copy() and update() will work in older versions.
Javascript Learn Solo: StackOverflow Questions
How to execute a program or call a system command?
Question by alan lai
How do you call an external command (as if I"d typed it at the Unix shell or Windows command prompt) from within a Python script?
Answer #1:
Use the subprocess module in the standard library:
The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes.
On Python 3.4 and earlier, use subprocess.call instead of .run:
subprocess.call(["ls", "-l"])
Answer #2:
Here"s a summary of the ways to call external programs and the advantages and disadvantages of each:
os.system("some_command with args") passes the command and arguments to your system"s shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example:
However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, et cetera. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs. See the documentation.
stream = os.popen("some_command with args") will do the same thing as os.system except that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don"t need to worry about escaping anything. See the documentation.
The Popen class of the subprocess module. This is intended as a replacement for os.popen, but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you"d say:
but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See the documentation.
The call function from the subprocess module. This is basically just like the Popen class and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example:
If you"re on Python 3.5 or later, you can use the new subprocess.run function, which is a lot like the above but even more flexible and returns a CompletedProcess object when the command finishes executing.
The os module also has all of the fork/exec/spawn functions that you"d have in a C program, but I don"t recommend using them directly.
The subprocess module should probably be what you use.
Finally, please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. There are serious security implications if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications consider this code:
and imagine that the user enters something "my mama didnt love me && rm -rf /" which could erase the whole filesystem.
Answer #3:
Typical implementation:
import subprocess
p = subprocess.Popen("ls", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
for line in p.stdout.readlines():
print line,
retval = p.wait()
You are free to do what you want with the stdout data in the pipe. In fact, you can simply omit those parameters (stdout= and stderr=) and it"ll behave like os.system().
Answer #4:
Some hints on detaching the child process from the calling one (starting the child process in background).
Suppose you want to start a long task from a CGI script. That is, the child process should live longer than the CGI script execution process.
The classical example from the subprocess module documentation is:
import subprocess
import sys
# Some code here
pid = subprocess.Popen([sys.executable, "longtask.py"]) # Call subprocess
# Some more code here
The idea here is that you do not want to wait in the line "call subprocess" until the longtask.py is finished. But it is not clear what happens after the line "some more code here" from the example.
My target platform was FreeBSD, but the development was on Windows, so I faced the problem on Windows first.
On Windows (Windows XP), the parent process will not finish until the longtask.py has finished its work. It is not what you want in a CGI script. The problem is not specific to Python; in the PHP community the problems are the same.
The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in Windows API.
If you happen to have installed pywin32, you can import the flag from the win32process module, otherwise you should define it yourself:
/* UPD 2015.10.27 @eryksun in a comment below notes, that the semantically correct flag is CREATE_NEW_CONSOLE (0x00000010) */
On FreeBSD we have another problem: when the parent process is finished, it finishes the child processes as well. And that is not what you want in a CGI script either. Some experiments showed that the problem seemed to be in sharing sys.stdout. And the working solution was the following:
I have not checked the code on other platforms and do not know the reasons of the behaviour on FreeBSD. If anyone knows, please share your ideas. Googling on starting background processes in Python does not shed any light yet.
Answer #5:
import os
os.system("your command")
Note that this is dangerous, since the command isn"t cleaned. I leave it up to you to google for the relevant documentation on the "os" and "sys" modules. There are a bunch of functions (exec* and spawn*) that will do similar things.
Javascript Learn Solo: StackOverflow Questions
How to frame two for loops in list comprehension python
I want to extract entries from entries when they are in tags:
result = []
for tag in tags:
for entry in entries:
if tag in entry:
result.extend(entry)
How can I write the two loops as a single line list comprehension?
Answer #1:
The best way to remember this is that the order of for loop inside the list comprehension is based on the order in which they appear in traditional loop approach. Outer most loop comes first, and then the inner loops subsequently.
So, the equivalent list comprehension would be:
[entry for tag in tags for entry in entries if tag in entry]
In general, if-else statement comes before the first for loop, and if you have just an if statement, it will come at the end. For e.g, if you would like to add an empty list, if tag is not in entry, you would do it like this:
[entry if tag in entry else [] for tag in tags for entry in entries]
Answer #2:
This should do it:
[entry for tag in tags for entry in entries if tag in entry]
Shop
Best laptop for Sims 4
$
Best laptop for Zoom
$499
Best laptop for Minecraft
$590
Best laptop for engineering student
$
Best laptop for development
$
Best laptop for Cricut Maker
$
Best laptop for hacking
$890
Best laptop for Machine Learning
$950
Latest questions
NUMPYNUMPY
psycopg2: insert multiple rows with one query
12 answers
NUMPYNUMPY
How to convert Nonetype to int or string?
12 answers
NUMPYNUMPY
How to specify multiple return types using type-hints
12 answers
NUMPYNUMPY
Javascript Error: IPython is not defined in JupyterLab
 points system , in which completing challenges and lessons rewards users with XP points that contribute to levels. Higher level programmers can display their XP level with digital certificates after completing the courses. </p> <p>In addition, while all the first lessons of a course are created from SoloLearn teachers, hundreds more are created by SoloLearn users who expand on a given language or subject . I am sure that different users have different styles of teaching, jumping from one lesson to another can be both interesting and frustrating.</p> <p>Since users are able to create content for SoloLearn, the amount of content available is staggering. After completing a course on a subject, users can browse advanced courses using the same language. While the quality of this content may vary, it’s nice to have an app with content that doesn’t appear to end. It allows users to continue learning more advanced topics, often presented by industry professionals . </p> <p>In addition, a large number of topics are available, from specific languages ‚Äã‚Äãto frameworks for design to fundamentals. Clearly getting that default content would take a long time. The app comes standard with courses in 13 different languages, not to mention the fundamentals, frameworks and community content lessons for all of the above</p> <p>Even after all of this is over, l L application is real. - life coding challenges (provided by both the app and the community) and games that allow users to test their skills before or after the end of classes. </p> <br>
<div id=)
