Python library versions
Sep. 27th, 2016 08:56 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
mtbcFor computers that I think of as being reliable production systems I tend to run OpenBSD if it's personal and the base installation largely suffices or Debian GNU/Linux otherwise. This is because each offers a series of stable releases (as opposed to a rolling release) to which they promptly backport important security fixes. So, I get to minimize security vulnerabilities while also avoiding many bugs: code only gets into a stable release after it has been tested for some time, in combination with the libraries upon which it depends, meaning that others have usually discovered and fixed the problems already and if they haven't then there are a lot of people who have exactly the same ecosystem running so reproducing bugs and sharing tips is much eased. Also, it is nice not to have to work with myriad packaging systems (CPAN, Hackage, etc. too) where one will do.
At work I write simple Python with much help from the official documentation and Stack Overflow. Already knowing Java, Perl and Haskell was a good starting point. I am not much familiar with the Python community nor with pip, although in terms of package management it hardly seems to be quite dpkg or xbps. Because of the above reasons, and with wanting to keep my work environment reliable, on my work computer I install our Python dependencies via Debian's own packaging of them. I am thus relying mostly on code that is widely used and tested while getting the frequent smaller changes that are truly warranted. I do use xbps-install -u on my personal laptop but if I thus break it a bit then I can live with the partial outage.
For OMERO our more Python-savvy people are moving toward an installation that uses virtualenv and pip that largely installs according to requirements like gunicorn>=19.3 with advice to use pip install --upgrade when upgrading OMERO to get the security updates for such dependencies. This obviously goes against my natural thinking in that different users may then get different combinations of library versions depending on when they upgrade, and possibly fairly infrequently given that OMERO itself can go for a long time without some urgent security release. However, I am not the one with experience in developing and deploying complex Python-based applications.
I thus wonder what is typical in the Python community and among those sysadmins who are responsible for the smooth running of the resulting software: if distribution-provided packaging is generally eschewed in favor of getting latest versions from pip and its ilk. Is there some security announcement list that generally covers pip's package repository? Has many years of Debian unduly conditioned me against whatever the latest released version of something happens to be? Are the main default repositories used by pip curated better than I realize? I don't know, but I'm curious to learn how my thinking is wrong in this case. I may be too conservative or have too little appreciation of the Python community's quality assurance efforts. It is worth noting that at work we have DevOps-style continuous integration servers frequently reinstalling and testing OMERO automatically so it may simply be that I don't give that extra safety net enough credit.
Update: My impression is that for installation on production servers we will largely be advising sysadmins to install the distribution-provided packaging of our software dependencies.
At work I write simple Python with much help from the official documentation and Stack Overflow. Already knowing Java, Perl and Haskell was a good starting point. I am not much familiar with the Python community nor with pip, although in terms of package management it hardly seems to be quite dpkg or xbps. Because of the above reasons, and with wanting to keep my work environment reliable, on my work computer I install our Python dependencies via Debian's own packaging of them. I am thus relying mostly on code that is widely used and tested while getting the frequent smaller changes that are truly warranted. I do use xbps-install -u on my personal laptop but if I thus break it a bit then I can live with the partial outage.
For OMERO our more Python-savvy people are moving toward an installation that uses virtualenv and pip that largely installs according to requirements like gunicorn>=19.3 with advice to use pip install --upgrade when upgrading OMERO to get the security updates for such dependencies. This obviously goes against my natural thinking in that different users may then get different combinations of library versions depending on when they upgrade, and possibly fairly infrequently given that OMERO itself can go for a long time without some urgent security release. However, I am not the one with experience in developing and deploying complex Python-based applications.
I thus wonder what is typical in the Python community and among those sysadmins who are responsible for the smooth running of the resulting software: if distribution-provided packaging is generally eschewed in favor of getting latest versions from pip and its ilk. Is there some security announcement list that generally covers pip's package repository? Has many years of Debian unduly conditioned me against whatever the latest released version of something happens to be? Are the main default repositories used by pip curated better than I realize? I don't know, but I'm curious to learn how my thinking is wrong in this case. I may be too conservative or have too little appreciation of the Python community's quality assurance efforts. It is worth noting that at work we have DevOps-style continuous integration servers frequently reinstalling and testing OMERO automatically so it may simply be that I don't give that extra safety net enough credit.
Update: My impression is that for installation on production servers we will largely be advising sysadmins to install the distribution-provided packaging of our software dependencies.
no subject
Date: 2016-09-28 03:01 pm (UTC)...but then, I am the grumpy sysadmin...
no subject
Date: 2016-10-03 08:49 pm (UTC)