The title of the page isn’t a hyperbole. If you have every tried to use Python beyond toy problems on Windows, you will have struggled with library clashes, 32/64 bit versions, ghost libraries (that seemed to be installed, but can’t be called).
As someone who uninstalled and reinstalled Python eight times in the last four weeks, let me tell you, getting Python working perfectly on Windows isn’t easy.
“But Mr T,” you say, “what’s the problem? You go to python.org and just download it. What’s the big deal?”
If you plan to just stick to solving Project Euler or other such toy problems, then the official site is good enough.
(Note: Lots of people have taken the previous sentence as an insult. It is not, and I apologise if it came out that way. All I meant was, in this day and age, it’s a shame there is no good packager for Python on Windows).
But I found that as soon as you try to do anything more complicated, like install libraries which need to be compiled (like numpy or scipy), the official build starts falling down. For my upcoming course on Python, I needed a lot of third party libraries, and I found that Python’s packaging system is really broken.
Here are some of the few Python versions I’ve tried:
1. The official from Python.org: Especially on Windows, this is good only for learning. I didn’t find it very stable, and even installing basic things like Virtualenv gave me problems.
2. The version from ActiveState: ActiveState have a version of Python, and it’s a lot more stable than the official version. It also makes it easy to install packages, and for a long time, this is what I used. But then I tried to install OpenCv, and got problems with versions. I decided to re-install Python, and that’s when I hit problems.
For some stupid reason, ActiveState install libraries in the C:\users\<name>\roaming\ region. These libraries take precedence over the local files (maybe due it being higher on the path hierarchy). Anyway, even after installing a different Python build (from Conda, see below), I kept getting library clashes which I couldn’t trace. I blamed Conda and spent weeks investigating obtuse error messages. I finally realised that ActiveState was to blame. The path to Python libraries must have been put in the registry or something, because even when I deleted the folders, it still looked for them in the Roaming folders.
I had to reinstall ActiveState Python, remove the libraries, and then uninstall it again.
Why am I telling you this? To show you how hard getting Python running can be on Windows. Recently, I discovered Anaconda, and that’s what I recommend now.
3. Anaconda
Based on the open source Conda packaging system, Anaconda is the best Python distribution I have found till now. For one, it comes with all the scientific libraries like numpy, scipy etc preinstalled, so you don’t have to worry about messing with compiling them yourself.
Second, it does not put important stuff in hidden folders (as far as I can see). Uninstalling it is easy and works.
Conda isn’t perfect, as I couldn’t get its environment system to work. But it’s a lot better than anything out there, and the only one I would recommend, at least on Windows
To install new packages, first try to use Conda install. If there is no Conda package, try pip, and then compiling from source (but never use easy_install).
In conclusion
Stop struggling with Python on Windows. Avoid distributions with fancy install methods (I’m looking at you, ActiveState). Install the Anaconda build, preferably 32 bit (as some libraries aren’t available for 64 bit, and you will get library clashes).
You describe my experience about 9 months ago! I’ve become so frustrated I am now getting myself a Mac – I have issues with Ruby, PHP and Node on Windows as well. The main program installers work without much bother but as soon as you start playing with third party libraries you will get issues.
I’ve not tried Anaconda but I’ve just had a look and it has an impressive set of libraries out the box, I’ll try it out. You might want to have a look at this site (if you haven’t already):
http://www.lfd.uci.edu/~gohlke/pythonlibs/
It’s an impressive up to date list of Windows binaries for Python packages and I’ve installed a few with good results!
Regards,
Matt 😉
Thanks Matt. Many people struggle, but there is a macho attitude in the community “Just be a big boy and get on with it”, which is why I decided to write this post.
It’s not just a Python issue- as you mention, other frameworks, most notably Ruby have had problems as well. I was looking at a job posting by a famous company for Ruby developers, and the advert basically said “You must be good with Linux, as Ruby doesn’t work on Windows” (their words, not mine).
Yes, I’ve seen that link, even tried a few libraries from it. While they work, my problem is the same: I’d rather have a centralised package manager than having to install/uninstall libraries from a dozen locations. That’s what I like about Conda. It makes it easier to manage once you have more than a dozen libraries.
Just a innocent question, (at least I’m trying not to make it loaded 🙂 ) why not install a virtual machine in windows for your use case, or dual boot to Linux, instead of buying another machine?
Robert, that’s a good point. For my course, I will be using a VM running Linux, but I’ve been asked by many people for help on Windows, so I thought I’d give it a try. I thought, how hard can it be? Plenty hard, it seems.
I did try Linux but I find you have to do a fair bit of heavy lifting with it. For example, Node has a Mac installer (and Windows) but you have to build it with Linux. That’s fine for a production server but when you want to tinker around with stuff (as I do) it gets a bit inconvenient. I use C# a lot and building that on Linux is a mission – on a Mac it just works out the box. Because Windows and Mac aren’t free means they get targeted for the professional dev tools. I know there are some lovers of Linux out there and I’m not knocking you!
To be clear, PHP does not have this much issue — wammp, xamp, winp etc. The easiest option moving forward is a linux VM as python seem to work much better on Linux.
Other alternative would be for Python to help provide common libraries that require compilation during the setup of python for windows.
Windows is the problem, not python.
Normally, i’d agree with you, but not in this case.
The scientific community has had long problems with the Python core developers, as described by the creator of Anaconda here:
http://technicaldiscovery.blogspot.co.uk/2013/12/why-i-promote-conda.html
With all due respect, the open-source bullshit philosophy is the ultimate problem. Why is that everything works perfect on windows except whatever open-source?
That’s a good point. There is a bit of passive aggressiveness when it comes to Windows.
Have you been developing for Windows for 1—2 weeks? For 7 years of Python programming I can assure you you should have no problems installing virtualenv, numpy, scikits or even PYAudio. Windows version of Python have the same stability as any other versions.
Don’t blame the tools.
Andrey, obviously, I’m pretty incompetent. I should stop showing my face in public. In fact, I’m going to do cover my face with a paper bag right now. 🙂
If you dont have vs installed already it is a pain to work out which sdk to install to get compiling working.
Last time I did numpy on windows in a virtualenv I had to unpack the installer and use easy install on the parts inside, which wasnt intuitive.
I didn’t blame any “tools” I said you’ll find a lot of third party libraries don’t run on Windows. Do you dispute this? And since I said I had problems months ago your question of whether I’ve only been using Windows for a couple of weeks I will have to put down to impertinence.
I use Anaconda on Linux and it works great! Python is not really suited for Windows but that’s why there are VMs for those who can’t get around needing windows on their machine.
That’s very true, Richard. That’s why I’m planning to stick with a Linux VM for my book
Anaconda might be the most powerful (and free!) software package for general end-user computing that there is. It just works, and it gives you everything you need on all three major platforms to do virtually anything out of the box.
I too recommend: http://www.lfd.uci.edu/~gohlke/pythonlibs/ for those Windows things you might need that are not available as conda packages (like Postgres and MySQL database drives to go with SQL Alchemy). The installers from there integrate well with anaconda (it will find all Python environments registered on the machine, let you pick one, and then install into the correct places).
anaconda installs in a single directory and makes it easy to have multiple versions and multiple python versions installed. It comes with the Spyder IDE, and all the iPython stuff, including what’s probably the most impressive component of the Python eco-system these days, the iPython Notebook interface. I now do basically all my calculation and ad-hoc computing in support of my development work inside Notebooks.
Honestly everyone who does any form of “computing” using their computer needs to have the anaconda system, and Python is the #1 language/tool to learn because you can do and drive basically anything and everything with it.
If you’re an iOS user, there are a couple great Python environments there too. Pythonista (http://omz-software.com/pythonista/) for the iPhone and iPad and now also the new Computable (http://computableapp.com/) which brings the iPython Notebook interface plus numpy, sympy, pandas, etc. to the iPad.
G.
Thanks Gavin, that was useful
If you want a proper setup – virtualenvs, and lots oc libraries that need a compiler it is a pain.
Having wasted many hours myself I always use linux in a vm if I have a windows box and advise others to do the same.
True
An interesting article and I recognise some of the challenges of Python on Windows but have to say that it doesn’t accord with my experience. I’ve found the ActiveState 32bit Python 2.7 to be stable and with pypm it is trivial to install 95% of what I need – matplotlib, numpy, scipy, ipython, pandas, nmap, etc, etc. OpenCV was easy to install too – the install instructions on the website work perfectly – you do read the instructions, don’t you? 😉 If you really want a command line package manager for the OS you can always install Chocolatey https://chocolatey.org/.
I’ve spent many years working with various Linux distros too and could write a similar article on the challenges posed by that OS. The time I’ve spent getting CUPS to work with my printer easily outweighs any time spent fiddling with Windows to get Python to work nicely.
I think that it’s quite realistic to develop with Python on Windows or *nix but you should expect in either OS that you sometimes have to get under the hood in order to get things to work!
ActiveState Python is stable, I just dont like its packaging manager.
And you are right that Linux has its own problems. But since a large number of people (majority?) use Windows, I thought I’d write on that
If you want to understand more about the python packaging problems and how conda is intended to be a complete solution to the problem, this might be worth 20m of your time…
https://www.youtube.com/watch?v=UaIvrDWrIWM
(actually all the SciPy 2014 talks are up on YouTube and they’re pretty much all awesome and worth your time)
G.
Generally developing Python on Windows isn’t too bad it’s just there are restrictions I’d rather do without.
I tried Chocolatey but got rid of it. Windows uninstalls have a habit of leaving turds in your file system and registry so I only install stuff with Revo Installer now, much safer 🙂
Windows uninstalls have a habit of leaving turds in your file system and registry
True. We need to start running Windows in a VM (instead of Linux, which is what people do), and every six months wipe everything and start clean 🙂
Any thoughts on clean uninstalling of Python ? I have 2 versions of python and dozens of libraries and I afraid simple uninstallation (through control panel) remains a mess behind.
Python installations should be independent. Delete the directories, and make sure you delete any path variables.
Also check under c:\users\USERNAME, some installers put stuff there.
I have a question or two:
Why Anaconda over some of the other SciPy stack offerings? I come from a MATLAB background and thus Python (x,y) looks nice to me. Ideally I would find something cross-platform (I use Linux, Mac, and Windows) which Python (x,y) is not. These are the offerings I’ve been considering:
Anaconda: https://www.anaconda.com/
Python (x,y): https://python-xy.github.io/
WinPython: https://winpython.github.io/
Pyzo: http://www.pyzo.org/
Even Anaconda is not straightforward with Windows in a corporate environment where things that are free is looked at with suspicion. Its a Nightmare when you attempt conda create -n pyxxx python=x.x.x only to result in a connection timeout error. Rather for winpython works for standard libraries like numpy, and scipy. But with matplotlib it crashes too.
Regards,
#0K
My sentiments exactly in regards to the painful process of getting Python to work on Windows 10.
I’m glad you talked about installing it 8 times not working because that was my next move. I suspected it wouldn’t make a difference so I am going to try some of the other suggestions here.
Thanks for the article.
Thanks for the post! I’ve been programming on both pylin and pywin for a number of few years now, bouncing between machines, vms, oses, and ides, virtualenvs / enviros for different projects. And yeah, it’s an elephant in the room for a mostly love-blinded python convert from 10 or so years back, but… I’ve kinda gotten used to the open-pit-toilet of the std pywin experience.
If my primary win box ever gains enough space i’ll be sure to give ana a crack, like I should have some time ago. But when you’re walking over a spaghetti bridge high above a croc-infested-turd-river filled with sharp rocks, the changes happen… slowly and carefully. 😉
Oh, and yes, your point is well made. Just because i do have wizard powers (including lightning from fingers) and have gotten through all the hellish hurdles to make things work, that doesn’t make me a clever-clogs because so little of the struggle was time well spent. Rather, it was time spent being an I-can’t-let-things-go-nerd, to my own detriment.
Shame, Mr Impractial-pants, shame.
hi all, i came here because I had a strange situation when working on anaconda with windows. At first everything worked perfectly but since i created a separate virtual env and loaded it to run a separate code from terminal in this new environment, I have been having issues running code on jupyter notebook. i was just not able to read a simple hdf file that returned a value error but the point is I cannot comprehend what happened and now after uninstalling and reinstalling anaconda, I am still having the same problem. I also noticed that after uninstalling anaconda, I still have the anaconda folder in my directory. I want to know if it is safe to manually delete it? Thanks.
Yes, delete it.
Also have a look at the anaconda forums- this might be a bug they know about. If not, you migth be able to raise a new one
For some reason, I’m not at all able to install python using Anaconda distribution on windows 7, 64 bit machine. IT gives error of “Conda not recognized …”
I’ve tried many different ways suggested on different forums. (diable ativirus,using miniconda etc.)
I’d really appreciate help on this …
Sachin Jadhav