What's the deal with CPython, Pypy, MicroPython, Jython...?
Wait, pypi and pypy are not the same thing?
Summary
When people say "Python", they usually mean CPython, the reference Python implementation. But there are others, oh, so many others:
Pypy, Python written in Python.
Stackless Python, the Eve Online secret sauce.
Jython, running on the JVM.
IronPython, running on .Net.
MicroPython, for micro-controllers.
Brython and Sculpt, generating JS from Python code.
RustPython, written in Rust, BTW.
Parrot and GraalVM, because they support everything.
Pyodide (and pyscript): compiling Python to WASM.
Unladen swallow, Cinder and Pyston: aka, FAANGS used Python a lot and now they need it to be faster
There are also a lot of things that seem, confusingly, like a new Python, but are not:
Nuitka: that's a Python transpiler. A good one.
WinPython, Anaconda and ActivePython: it's 3 CPython in a commercial trench coat.
Psyco and Pyjion: they are pluggable JIT for CPython.
You probably shouldn't care though. Just knowing CPython is fine.
If you really want to remember a few more, pypy, microPython and pyodide are probably the most important ones to know about, right now.
What's in a name?
Jargon is a real struggle when you learn a new field, and this is particularly true with programming, which is mostly a made of conventions on top of conventions.
Conventions for byte order, for protocol workflow, for keywords meaning...
But the worst offender is the ecosystem made by the community, because we like clever names, puns and silly acronyms. Since the hive mind hasn't blessed us with peace yet, there is no namespace the human brain collectively shares. We kinda do the cache invalidation of the naming thingy at the scale of the planet manually.
So the libs, frameworks and websites have terrible names.
Now it turns out Python as a language has been initially coded in C by Guido van Rossum. Because of its success, many other projects have recoded Python in other languages or with different goals in mind.
And each of those projects, alternative implementations really, needed a name to be distinguished from the original.
That's how colorful names like CPython, Pypy, MicroPython, Iron Python, Jython, and Pyston, came around.
Let's see what is what.
You likely use CPython
When people talk about Python, 99% of the time, they mean CPython, the reference Python implementation written in C. That's the one from van Rossum, the one you download from python.org, the one you install with apt
, etc.
In fact, you can stop reading the article right now and be happy with this information alone. It's enough. Most devs will never, ever have to use another version of Python. Or even talk about it, except to brag at the coffee machine...err... on the informal team chat.
However, if you are curious, wants to expand your horizon, or have a very specific niche requirement, maybe we can interest you in some exotic goods.
The many snakes in the pit
Pypy
Pypy is the most famous CPython alternative. It's Python written... in Python. Well, in RPython, a subset of the language. Its main feature is to be faster than CPython, mainly thanks to Just-in-time compilation.
It is slower to start than CPython, but if it runs for long enough, it will become faster and faster. This makes it particularly useful for long-running processes, such as daemons, task queues and web servers.
On the other hand, it is incompatible with a many c-extensions out there, and they will either not work, or be slower than usual.
Stackless Python
Stackless is a version of Python created to improve its performances and network concurrency capabilities, back where in 2, it didn't have asyncio.
Many improvements in Stackless Python have been extracted and found their way into Pypy (for perfs) or some libraries like gevent (for concurrency).
Nowadays, it is considered mostly a curiosity, and unless you are working on the Eve Online video game, you are unlikely to be ever using it.
Jython
Jython is Python for the Java Virtual Machine. It can run from inside Java code, and even import and use Java classes.
The main benefit is the compatibility between the two ecosystems. However, it is still stuck in Python 2.
IronPython
IronPython is an implementation of Python for the .NET framework. It can use .NET framework libraries and can also expose Python code to other languages, such as C# or F#.
Just like Jython, being able to benefit from both toolbox is the appeal, if you can limit yourself to Python 3.4, and accept some incompatibility.
MicroPython
MicroPython is a lean and efficient implementation of Python 3 that includes a small subset of the Python standard library and is optimized to run on micro-controllers and in constrained environments.
It has many of the Python qualities, yet is compact enough to fit and run within just 256k of code space and 16k of RAM. Yes, K.
It can run on those teeny-weeny boards, and even Arduino, which makes for fun projects, without the need to reach for a full-blown raspi.
The compromise, of course, is that there are parts of Python you cannot use.
Brython and Sculpt
Javascript having a virtual monopoly in the browser, it was only a matter of type before someone tried to code Python in JS. Browser Python, or Brython, and Sculpt are born out of this effort.
They support a quite impressive set of Python 3 constructs. Although they will cost you an initial download for the run time, generate quite some code, and don't let you pip install things. On top of this, they have some overhead.
All in all, I would advise professionals interested in frontend dev to just bite the bullet and learn JS. The language has improved a lot over the years. Plus it's the easiest way to get decent perfs, and less trouble debugging. And if you really don't want to, htmx and alpine.js are your friends.
Alternatively, if you are not a professional dev, dash is probably a better shortcut.
Those tools are good for education purposes, though.
There are alternatives like Transcrypt, a AoT transpiler that turns Python AST into JS, and RapydScript, an alternative to CoffeeScript with a Python-like syntax.
RustPython
RustPython is an experimental Python interpreter written in the Rust programming language. It comes with the ability to call Python from Rust and vice versa, to compile the program to WASM and even feature a small JIT.
Python and Rust are getting quite good friends these days, so it has to compete with less radical solutions, like Maturin being the new favorite toy to build Python compiled extensions.
Nevertheless, an interesting experiment. After all, look at what Pypy did.
It's quite young, so don't expect it to be production ready,
Multi-language VM
There are very ambitious projects out there, like Parrot and GraalVM, that aim to support multiple languages at once.
Those two include Python in the languages they support. However, I must confess I don't know much about them, I never tried them, nor do I know anybody that uses them professionally.
So that's about as much as I can say without repeating verbatim what the docs state.
Faster Python
Python being very slow, several projects have attempted to speed up faster over the years, way before Microsoft hired our ex benevolent dictator for life to work on the topic.
The first one I can remember was unladen swallow (at last a good Monty Python joke), targeting Python 2.6. It was a project led by a Google intern, that left us with a particularly honest retrospective. It's funny to think at the time, the demand for a faster Python was not strong enough to give it momentum.
Since Python is used everywhere, other big players gave it a try.
Facebook attempt was named Cinder, and it powers Instagram, since it started as a Django app. It features bytecode caching, a JIT, coroutine eager evaluation, and other tricks, that is inspiring the current CPython development. A good thing since they clearly says it's "not polished or documented for anyone else's use", so you probably don't want to use it as-is.
Dropbox being initially coded with a lot of Python also gave it a try, and Pyston was born. It is now a stand-alone project that should be a drop-in replacement for python 3.7 to 3.10, with a 35-65% speedup. It does break API compat, so no c-extension wheels for you: you must compile them.
However, their biggest success so far is pyston_lite_autoload, which is a package you can install to get 10% by just installing it. No need for a new Python.
(Warning: Pyston is not to be confused with piston, a JS project, and pyston, the name of the repository for aiopyston, the lib to talk to piston. Didn't I say we suck at naming?)
Not a Python implementation
Pypi
Pypi is the website from which pip
downloads the packages you install. It's not to be confused with Pypy, the Python implementation written in Python. Pypi is not a Python implementation.
Pyodide and Pyscript
Pyodide is technically CPython, but ported to web assembly. This allows to have the full Python experience, including pip installing things and using Numpy or metaclasses, in the Web browser. It even has a virtual file system, so you can create files or SQLite databases!
For all this goodness, the first download is heavy, and execution is slow, but that's the closest thing you get from a zero-install Python.
I really like this Jupyter notebook demo. You click on the link, you open a website, you wait a little, and bam, you have a full Python env ready to play with.
This runs 100% on your machine, you can hammer it, the server will not care.
Recently, there was some noise about Pyscript, which promised Python scripting in the web browser.
It is based on Pyodide, with some sugar to make it work as if it was native to a web page.
Because Anaconda is pushing for it, it has been on the spotlight despite having a weird value proposition, as it's much heavier than, say, Brython, which has pretty much the same user story.
The difference being you can use a real CPython with pip and a fake file system, but it's very hard to justify the huge price of it for interactivity: it's almost 3Mb to load, for the hello world.
It's great for teaching, though, not to mention there are experiments with MicroPython and Pyodide that could make it more interesting.
Cython and Numba
Cython and Numba are not really Python implementations, but rather specialized Python compilers. If you write Python in a certain way, you can use Cython to compile Python ahead of time or Numba to compile it just in time, and get much better performances.
On some algo, you can expect c-like performances with Python-looking code.
I call them specialized compilers, because you usually don't need to compile your entire Python program, only the parts that you know are slow and that those tools accelerate well.
E.G: Numba is pretty good at accelerating big loops of calculations with floats, something Python is notoriously bad at.
Recently, a contender appeared: Mojo. It's not clear if Mojo is a separate language implementation that features a superset of Python, or a specialized compiler like Cython and Numba. It's proprietary and runs only on their private platform, so it's hard to know for sure. It got some buzz recently, so I include it in there, although I'm going to wait until they make it more open to invest any time in it.
Nuitka
Nuitka is a full-blown Python transpiler, that turns your Python 2.6 (!) to 3.11 program into C, and compile that into a stand-alone executable you can ship to your clients. They don't even need to know you wrote the software in Python, if you hide the stack trace.
It is an excellent project, which has been quietly doing very serious work for years. The compatibility is strong: it handled all the code I could feed it with, and even deals with the scientific stacks, GUIs and data files.
While it's not as easy to compile as with Rust or Go, Nuitka provides quite a streamlined experience. If you have a compiler installed, it's usually a single command to get a beautiful executable.
It's my go-to tool to create end-user programs. I dream of it being coupled with Zig to provide cross-compilation, and some tool to create a nice installer.
They have a commercial offering to offer reverse-engineering protection, including stack trace + constant + data file encryption, etc.
Ok, the article now reads like an ad for Nuitka, but it always surprises me how good it is, and little it is known.
WinPython, Anaconda and ActivePython
Those are commercial Python distributions, not implementations. They use the regular CPython, and provide them in a certain format. WinPython provides Scipy and Spyder without the need to install things (it's a .zip), Anaconda has a strong commercial offer for support and infra targeting big companies and ActivePython gives extended support for obsolete versions of Python, like 2.7.
Anaconda is by far the most popular one, because it solved a lot of packaging problems before we got wheels. Also, they have very good corporate integration, something the FOSS world usually neglects and wonders why LDAP and Jira are still around.
But if you read the blog, I now recommend avoiding it if you can.
It's not always possible, and I regularly have to work with it, which gives me regular confirmation that it brings as many problems as it solves.
Pluggable JIT
Instead of trying to recode an entire Python, some projects tries to speed it up by providing just-in-time compilation to the existing CPython interpreter. That's a what pyston_lite_autoload ended up to be, despite the fact that Pyston started as a full reimplementation.
Psyco used to do exactly that, but for Python 2. It's now dead, but I remember fondly being able to just do import psyco; psyco.full()
and get my program run faster magically. Armin Rigo, psyco’s author, eventually became a founder of pypy.
Lastly, we got Pyjion, a pip installable Python 3.10 booster that uses the .NET 7 CLR JIT under the hood. An interesting idea, but not very active for now.
And so many others
For a short time, there was a project named HotPy that was promising a new VM with a tracing JIT, and even a hotpy 2 that I can't find anymore.
The naming adventure continues with Batavia, a project to port Python on the Web, from the promising Beeware that got the Carbonite treatment.
Pypy.js, which is exactly what you think it is, and froze my browser tab.
Coconut, that is a functional language that compiles to python bytecode.
Hy, a Lisp dialect embeded in Python.
And I must forget many others.
So next time somebody says "they should make Python better, faster, stronger", point them to this article. Because whoever “they” are, they are trying.
Thanks for that very interesting article!
I'm a total Python lover for around 4-5 years. You have showed me now some interesting stuff that I didn't know about.
While I only skimmed over most parts of your article, I immediately stopped at the "nuitka" part. Because I used nuitka some years ago and really liked it. Interesting now that you are a big fan of it.
Do you really use it for shipping end user apps? Because I've actually never dealt with packaging for end users, I just have always needed scripts for myself till now. But if nuitka is really a good way to ship end user projects, then actually I already know how to do packaging if I'll need it one day. Did you never had problems with nuitka? Just curious now.
And is nuitka really only able to use Python 3.1 and nothing newer? Wasn't aware of that. Might be also problematic if you use newer Python syntax. Why isn't it supporting newer Python versions?
BIG EDIT:
Sorry you meant Python 3.11, I was reading that like Python 3.1.1 lol. Well, it's a bit late, must take some sleep soon 🙂 Happy to see that nuitka is totally up-to-date concerning Python versions.
Cheers!
Transcrypt is actually a Python to JavaScript transpiler. It uses the Python AST module under the hood during the transpile process. So not only is it not just "Python-like", the Python code it processes *has* to be syntax correct Python for it to even work. The end result is JS code that can run natively in a web browser that only needs an additional 70K JS file at run time. It is intended to allow Python developers to utilize the JS ecosystem. I personally use it to develop React/MUI applications with Python.
The main difference between Brython/Skulpt and Transcrypt, is that Transcrypt has an ahead-of-time compile step whereas the others transpile in the browser on page load or at run time.