I have a
requirements.txt file that I"m using with Travis-CI. It seems silly to duplicate the requirements in both
setup.py, so I was hoping to pass a file handle to the
install_requires kwarg in
Is this possible? If so, how should I go about doing it?
Here is my
guessit>=0.5.2 tvdb_api>=1.8.2 hachoir-metadata>=1.3.3 hachoir-core>=1.3.3 hachoir-parser>=1.3.4
You can flip it around and list the dependencies in
setup.py and have a single character — a dot
. — in
Alternatively, even if not advised, it is still possible to parse the
requirements.txt file (if it doesn"t refer any external requirements by URL) with the following hack (tested with
install_reqs = parse_requirements("requirements.txt", session="hack")
This doesn"t filter environment markers though.
In old versions of pip, more specifically older than 6.0, there is a public API that can be used to achieve this. A requirement file can contain comments (
#) and can include some other files (
-r). Thus, if you really want to parse a
requirements.txt you can use the pip parser:
from pip.req import parse_requirements # parse_requirements() returns generator of pip.req.InstallRequirement objects install_reqs = parse_requirements(<requirements_path>) # reqs is a list of requirement # e.g. ["django==1.5.1", "mezzanine==1.4.6"] reqs = [str(ir.req) for ir in install_reqs] setup( ... install_requires=reqs )
On the face of it, it does seem that
setup.py are silly duplicates, but it"s important to understand that while the form is similar, the intended function is very different.
The goal of a package author, when specifying dependencies, is to say "wherever you install this package, these are the other packages you need, in order for this package to work."
In contrast, the deployment author (which may be the same person at a different time) has a different job, in that they say "here"s the list of packages that we"ve gathered together and tested and that I now need to install".
The package author writes for a wide variety of scenarios, because they"re putting their work out there to be used in ways they may not know about, and have no way of knowing what packages will be installed alongside their package. In order to be a good neighbor and avoid dependency version conflicts with other packages, they need to specify as wide a range of dependency versions as can possibly work. This is what
The deployment author writes for a very different, very specific goal: a single instance of an installed application or service, installed on a particular computer. In order to precisely control a deployment, and be sure that the right packages are tested and deployed, the deployment author must specify the exact version and source-location of every package to be installed, including dependencies and dependency"s dependencies. With this spec, a deployment can be repeatably applied to several machines, or tested on a test machine, and the deployment author can be confident that the same packages are deployed every time. This is what a
So you can see that, while they both look like a big list of packages and versions, these two things have very different jobs. And it"s definitely easy to mix this up and get it wrong! But the right way to think about this is that
requirements.txt is an "answer" to the "question" posed by the requirements in all the various
setup.py package files. Rather than write it by hand, it"s often generated by telling pip to look at all the
setup.py files in a set of desired packages, find a set of packages that it thinks fits all the requirements, and then, after they"re installed, "freeze" that list of packages into a text file (this is where the
pip freeze name comes from).
So the takeaway:
setup.pyshould declare the loosest possible dependency versions that are still workable. Its job is to say what a particular package can work with.
requirements.txtis a deployment manifest that defines an entire installation job, and shouldn"t be thought of as tied to any one package. Its job is to declare an exhaustive list of all the necessary packages to make a deployment work.
Cloud computing provides the capability to use computing and storage resources on a metered basis and reduce the investments in an organization’s computing infrastructure. The spawning and deletion ...
Pandas for Everyone: Python Data Analysis (Addison-Wesley Data & Analytics Series), 1st Edition. Pandas for Everyone brings together the practical knowledge and insights you need to solve real-worl...
We are witnessing a movement that will completely transform any part of business and society. The word we have given to this move- ment is Big Data and it will change everything, from the way banks an...
This book project was first presented to me during my first week in my current role of managing the data mining development at SAS. Writ- ing a book has always been a bucket‐list item, and I was ver...