Python Requests and persistent sessions

ellipsis | StackOverflow

I am using the requests module (version 0.10.0 with Python 2.5). I have figured out how to submit data to a login form on a website and retrieve the session key, but I can"t see an obvious way to use this session key in subsequent requests. Can someone fill in the ellipsis in the code below or suggest another approach?

>>> import requests
>>> login_data =  {"formPosted":"1", "login_email":"[email protected]", "password":"pw"}
>>> r = requests.post("https://localhost/login.py", login_data)
>>> 
>>> r.text
u"You are being redirected <a href="profilePage?_ck=1349394964">here</a>"
>>> r.cookies
{"session_id_myapp": "127-0-0-1-825ff22a-6ed1-453b-aebc-5d3cf2987065"}
>>> 
>>> r2 = requests.get("https://localhost/profile_data.json", ...)

Answer rating: 250

You can easily create a persistent session using:

s = requests.Session()

After that, continue with your requests as you would:

s.post("https://localhost/login.py", login_data)
#logged in! cookies saved for future requests.
r2 = s.get("https://localhost/profile_data.json", ...)
#cookies sent automatically!
#do whatever, s will keep your cookies intact :)

For more about sessions: https://requests.kennethreitz.org/en/master/user/advanced/#session-objects





Python Requests and persistent sessions: StackOverflow Questions

What does the Ellipsis object do?

While idly surfing the namespace I noticed an odd looking object called Ellipsis, it does not seem to be or do anything special, but it"s a globally available builtin.

After a search I found that it is used in some obscure variant of the slicing syntax by Numpy and Scipy... but almost nothing else.

Was this object added to the language specifically to support Numpy + Scipy? Does Ellipsis have any generic meaning or use at all?

D:workspace
umpy>python
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> Ellipsis
Ellipsis

What do ellipsis [...] mean in a list?

I was playing around in python. I used the following code in IDLE:

p  = [1, 2]
p[1:1] = [p]
print p

The output was:

[1, [...], 2]

What is this […]? Interestingly I could now use this as a list of list of list up to infinity i.e.

p[1][1][1]....

I could write the above as long as I wanted and it would still work.

EDIT:

  • How is it represented in memory?
  • What"s its use? Examples of some cases where it is useful would be helpful.
  • Any link to official documentation would be really useful.

How do you use the ellipsis slicing syntax in Python?

Question by miracle2k

This came up in Hidden features of Python, but I can"t see good documentation or examples that explain how the feature works.

Answer #1

typing.Tuple and typing.List are Generic types; this means you can specify what type their contents must be:

def f(points: Tuple[float, float]):
    return map(do_stuff, points)

This specifies that the tuple passed in must contain two float values. You can"t do this with the built-in tuple type.

typing.Tuple is special here in that it lets you specify a specific number of elements expected and the type of each position. Use ellipsis if the length is not set and the type should be repeated: Tuple[float, ...] describes a variable-length tuple with floats.

For typing.List and other sequence types you generally only specify the type for all elements; List[str] is a list of strings, of any size. Note that functions should preferentially take typing.Sequence as arguments and typing.List is typically only used for return types; generally speaking most functions would take any sequence and only iterate, but when you return a list, you really are returning a specific, mutable sequence type.

You should always pick the typing generics even when you are not currently restricting the contents. It is easier to add that constraint later with a generic type as the resulting change will be smaller.

Answer #2

As of Python 3.5 and PEP484, the literal ellipsis is used to denote certain types to a static type checker when using the typing module.

Example 1:

Arbitrary-length homogeneous tuples can be expressed using one type and ellipsis, for example Tuple[int, ...]

Example 2:

It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis (three dots) for the list of arguments:

def partial(func: Callable[..., str], *args) -> Callable[..., str]:
    # Body

Answer #3

While the proposed duplicate What does the Python Ellipsis object do? answers the question in a general python context, its use in an nditer loop requires, I think, added information.

https://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#modifying-array-values

Regular assignment in Python simply changes a reference in the local or global variable dictionary instead of modifying an existing variable in place. This means that simply assigning to x will not place the value into the element of the array, but rather switch x from being an array element reference to being a reference to the value you assigned. To actually modify the element of the array, x should be indexed with the ellipsis.

That section includes your code example.

So in my words, the x[...] = ... modifies x in-place; x = ... would have broken the link to the nditer variable, and not changed it. It"s like x[:] = ... but works with arrays of any dimension (including 0d). In this context x isn"t just a number, it"s an array.

Perhaps the closest thing to this nditer iteration, without nditer is:

In [667]: for i, x in np.ndenumerate(a):
     ...:     print(i, x)
     ...:     a[i] = 2 * x
     ...:     
(0, 0) 0
(0, 1) 1
...
(1, 2) 5
In [668]: a
Out[668]: 
array([[ 0,  2,  4],
       [ 6,  8, 10]])

Notice that I had to index and modify a[i] directly. I could not have used, x = 2*x. In this iteration x is a scalar, and thus not mutable

In [669]: for i,x in np.ndenumerate(a):
     ...:     x[...] = 2 * x
  ...
TypeError: "numpy.int32" object does not support item assignment

But in the nditer case x is a 0d array, and mutable.

In [671]: for x in np.nditer(a, op_flags=["readwrite"]):
     ...:     print(x, type(x), x.shape)
     ...:     x[...] = 2 * x
     ...:     
0 <class "numpy.ndarray"> ()
4 <class "numpy.ndarray"> ()
...

And because it is 0d, x[:] cannot be used instead of x[...]

----> 3     x[:] = 2 * x
IndexError: too many indices for array

A simpler array iteration might also give insight:

In [675]: for x in a:
     ...:     print(x, x.shape)
     ...:     x[:] = 2 * x
     ...:     
[ 0  8 16] (3,)
[24 32 40] (3,)

this iterates on the rows (1st dim) of a. x is then a 1d array, and can be modified with either x[:]=... or x[...]=....

And if I add the external_loop flag from the next section, x is now a 1d array, and x[:] = would work. But x[...] = still works and is more general. x[...] is used all the other nditer examples.

In [677]: for x in np.nditer(a, op_flags=["readwrite"], flags=["external_loop"]):
     ...:     print(x, type(x), x.shape)
     ...:     x[...] = 2 * x
[ 0 16 32 48 64 80] <class "numpy.ndarray"> (6,)

Compare this simple row iteration (on a 2d array):

In [675]: for x in a:
     ...:     print(x, x.shape)
     ...:     x[:] = 2 * x
     ...:     
[ 0  8 16] (3,)
[24 32 40] (3,)

this iterates on the rows (1st dim) of a. x is then a 1d array, and can be modified with either x[:] = ... or x[...] = ....

Read and experiment with this nditer page all the way through to the end. By itself, nditer is not that useful in python. It does not speed up iteration - not until you port your code to cython.np.ndindex is one of the few non-compiled numpy functions that uses nditer.

Answer #4

In Python, you can do:

test = float("inf")

In Python 3.5, you can do:

import math
test = math.inf

And then:

test > 1
test > 10000
test > x

Will always be true. Unless of course, as pointed out, x is also infinity or "nan" ("not a number").

Additionally (Python 2.x ONLY), in a comparison to Ellipsis, float(inf) is lesser, e.g:

float("inf") < Ellipsis

would return true.

Answer #5

This came up in another question recently. I"ll elaborate on my answer from there:

Ellipsis is an object that can appear in slice notation. For example:

myList[1:2, ..., 0]

Its interpretation is purely up to whatever implements the __getitem__ function and sees Ellipsis objects there, but its main (and intended) use is in the numpy third-party library, which adds a multidimensional array type. Since there are more than one dimensions, slicing becomes more complex than just a start and stop index; it is useful to be able to slice in multiple dimensions as well. E.g., given a 4x4 array, the top left area would be defined by the slice [:2,:2]:

>>> a
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12],
       [13, 14, 15, 16]])

>>> a[:2,:2]  # top left
array([[1, 2],
       [5, 6]])

Extending this further, Ellipsis is used here to indicate a placeholder for the rest of the array dimensions not specified. Think of it as indicating the full slice [:] for all the dimensions in the gap it is placed, so for a 3d array, a[...,0] is the same as a[:,:,0] and for 4d, a[:,:,:,0], similarly, a[0,...,0] is a[0,:,:,0] (with however many colons in the middle make up the full number of dimensions in the array).

Interestingly, in python3, the Ellipsis literal (...) is usable outside the slice syntax, so you can actually write:

>>> ...
Ellipsis

Other than the various numeric types, no, I don"t think it"s used. As far as I"m aware, it was added purely for numpy use and has no core support other than providing the object and corresponding syntax. The object being there didn"t require this, but the literal "..." support for slices did.

Answer #6

Enumerating the possibilities allowed by the grammar:

>>> seq[:]                # [seq[0],   seq[1],          ..., seq[-1]    ]
>>> seq[low:]             # [seq[low], seq[low+1],      ..., seq[-1]    ]
>>> seq[:high]            # [seq[0],   seq[1],          ..., seq[high-1]]
>>> seq[low:high]         # [seq[low], seq[low+1],      ..., seq[high-1]]
>>> seq[::stride]         # [seq[0],   seq[stride],     ..., seq[-1]    ]
>>> seq[low::stride]      # [seq[low], seq[low+stride], ..., seq[-1]    ]
>>> seq[:high:stride]     # [seq[0],   seq[stride],     ..., seq[high-1]]
>>> seq[low:high:stride]  # [seq[low], seq[low+stride], ..., seq[high-1]]

Of course, if (high-low)%stride != 0, then the end point will be a little lower than high-1.

If stride is negative, the ordering is changed a bit since we"re counting down:

>>> seq[::-stride]        # [seq[-1],   seq[-1-stride],   ..., seq[0]    ]
>>> seq[high::-stride]    # [seq[high], seq[high-stride], ..., seq[0]    ]
>>> seq[:low:-stride]     # [seq[-1],   seq[-1-stride],   ..., seq[low+1]]
>>> seq[high:low:-stride] # [seq[high], seq[high-stride], ..., seq[low+1]]

Extended slicing (with commas and ellipses) are mostly used only by special data structures (like NumPy); the basic sequences don"t support them.

>>> class slicee:
...     def __getitem__(self, item):
...         return repr(item)
...
>>> slicee()[0, 1:2, ::5, ...]
"(0, slice(1, 2, None), slice(None, None, 5), Ellipsis)"

Answer #7

If you have a list of hashable objects (filenames would probably be strings, so they should count):

lst = ["foo.py", "bar.py", "baz.py", "qux.py", Ellipsis]

you can construct the set directly:

s = set(lst)

In fact, set will work this way with any iterable object! (Isn"t duck typing great?)


If you want to do it iteratively:

s = set()
for item in iterable:
    s.add(item)

But there"s rarely a need to do it this way. I only mention it because the set.add method is quite useful.

Answer #8

In Python 3, you can¹ use the Ellipsis literal ... as a “nop” placeholder for code that hasn"t been written yet:

def will_do_something():
    ...

This is not magic; any expression can be used instead of ..., e.g.:

def will_do_something():
    1

(Can"t use the word “sanctioned”, but I can say that this use was not outrightly rejected by Guido.)

¬π "can" not in {"must", "should"}

Answer #9

The ellipsis is used in numpy to slice higher-dimensional data structures.

It"s designed to mean at this point, insert as many full slices (:) to extend the multi-dimensional slice to all dimensions.

Example:

>>> from numpy import arange
>>> a = arange(16).reshape(2,2,2,2)

Now, you have a 4-dimensional matrix of order 2x2x2x2. To select all first elements in the 4th dimension, you can use the ellipsis notation

>>> a[..., 0].flatten()
array([ 0,  2,  4,  6,  8, 10, 12, 14])

which is equivalent to

>>> a[:,:,:,0].flatten()
array([ 0,  2,  4,  6,  8, 10, 12, 14])

In your own implementations, you"re free to ignore the contract mentioned above and use it for whatever you see fit.

Answer #10

Python has the syntactical requirement that code blocks (after if, except, def, class etc.) cannot be empty. Empty code blocks are however useful in a variety of different contexts, such as in examples below, which are the most frequent use cases I have seen.

Therefore, if nothing is supposed to happen in a code block, a pass is needed for such a block to not produce an IndentationError. Alternatively, any statement (including just a term to be evaluated, like the Ellipsis literal ... or a string, most often a docstring) can be used, but the pass makes clear that indeed nothing is supposed to happen, and does not need to be actually evaluated and (at least temporarily) stored in memory.

  • Ignoring (all or) a certain type of Exception (example from xml):

     try:
         self.version = "Expat %d.%d.%d" % expat.version_info
     except AttributeError:
         pass # unknown
    

    Note: Ignoring all types of raises, as in the following example from pandas, is generally considered bad practice, because it also catches exceptions that should probably be passed on to the caller, e.g. KeyboardInterrupt or SystemExit (or even HardwareIsOnFireError – How do you know you aren"t running on a custom box with specific errors defined, which some calling application would want to know about?).

     try:
         os.unlink(filename_larry)
     except:
         pass
    

    Instead using at least except Error: or in this case preferably except OSError: is considered much better practice. A quick analysis of all Python modules I have installed gave me that more than 10% of all except ...: pass statements catch all exceptions, so it"s still a frequent pattern in Python programming.

  • Deriving an exception class that does not add new behaviour (e.g., in SciPy):

     class CompileError(Exception):
         pass
    

    Similarly, classes intended as abstract base class often have an explicit empty __init__ or other methods that subclasses are supposed to derive (e.g., pebl):

     class _BaseSubmittingController(_BaseController):
         def submit(self, tasks): pass
         def retrieve(self, deferred_results): pass
    
  • Testing that code runs properly for a few test values, without caring about the results (from mpmath):

     for x, error in MDNewton(mp, f, (1,-2), verbose=0,
                              norm=lambda x: norm(x, inf)):
         pass
    
  • In class or function definitions, often a docstring is already in place as the obligatory statement to be executed as the only thing in the block. In such cases, the block may contain pass in addition to the docstring in order to say ‚ÄúThis is indeed intended to do nothing.‚Äù, for example in pebl:

     class ParsingError(Exception):
         """Error encountered while parsing an ill-formed datafile."""
         pass
    
  • In some cases, pass is used as a placeholder to say ‚ÄúThis method/class/if-block/... has not been implemented yet, but this will be the place to do it‚Äù, although I personally prefer the Ellipsis literal ... in order to strictly differentiate between this and the intentional ‚Äúno-op‚Äù in the previous example. (Note that the Ellipsis literal is a valid expression only in Python 3)

    For example, if I write a model in broad strokes, I might write

     def update_agent(agent):
         ...
    

    where others might have

     def update_agent(agent):
         pass
    

    before

     def time_step(agents):
         for agent in agents:
             update_agent(agent)
    

    as a reminder to fill in the update_agent function at a later point, but run some tests already to see if the rest of the code behaves as intended. (A third option for this case is raise NotImplementedError. This is useful in particular for two cases: Either “This abstract method should be implemented by every subclass, and there isn"t a generic way to define it in this base class”, or “This function, with this name, is not yet implemented in this release, but this is what its signature will look like”)

Get Solution for free from DataCamp guru