Qt for Python:  the 6.4.0.1 release, what?

We regret to inform you that during the 6.4.0 release, we were attacked by release goblins.

If you are not familiar with them, release goblins (dimittis coboli) are strange small creatures that can make you forget things, change code values, and even boycott new releases! we were safe for a long time, but due to the popularity we have been seeing on the new releases, we were victims of them.

But we have two good news, we have trained the whole team on a TDAG (Tactical Defense Against Goblins) course, but more importantly, a new release is available.

We had the option to wait until 6.4.1, but we didn't want to keep you hanging in case you are one of the lucky people that is already using Python 3.11.

What happened? 😨

As soon as we knew Python 3.11 was stable enough to start testing it, the whole team spent time trying to tackle all the issues we found, among them:

• the usage of intern strings (Check PyUnicode_InternFromString)
• adapt the new Enum implementation with 3.11 to avoid duplication.
• use the 3.11 buffer interface for our limited API implementation
• fix the garbage collector tracking for properties
• even we provided a new ad-hoc internal function to debug frames

Everything was going well, and we felt really happy to have Python 3.11 support even with the RC2.

...but, we forgot a tiny little detail: while changing all the 3.10 restrictions, there was one line we forgot to update from our setup.cfg which is super important to allow people to install those wheels.  This line:python_requires = >=3.7, <3.11 where as you might have noticed, we needed to change <3.11 to <3.12

Once that was in, we noticed that also we had a few issues with our new shiny deployment wrapper, that in some configurations was not properly working. We used those extra days to check more and more corners that might be a bit broken, and we found a couple of issues in some examples, and adapted the wheel name to include cp37-abi3 instead of cp36-abi3, because 6.4 is the first Qt for Python version to drop support for Python 3.6, as announced in previous releases.

Do you have any idea what is that abi3 thing? are you curious about how we package the project? don't worry, since you are here, let us explore it:

The Python Limited API 📚

You might have been noticing that some Python project provide packages (wheels) for specific Python versions, for example the famous NumPy project has individuals wheels for Python 3.8, 3.9, 3.10 and 3.11 (per platform) meaning many files, but we have only one per platform. How is that possible?

This doesn't mean the packaging is better or worse, only more convenient!

The Limited API is a subset of the Python's C-API, it was introduced in Python 3.2 with the main goal of providing an option for backwards compatibility. In simple words, this only mean that if I compile code for Python 3.9.7 it will work on 3.8.x, 3.7.x, etc. Read more about it here.

Together with the abi3 tag, which is the CPython stable ABI tag for pip to understand the compatibility of the package, one can generate packages with the cp3X-abi3 tag meaning that will be supported from cp3X onward.

In our CI, we build, package and test PySide and Shiboken for different Python versions and OS combinations, but we didn't have yet a test to install the wheels on the newly 3.11 - mainly because at the time of the 6.4.0 release, Python 3.11 was not officially released, and we only used the RC2 for local development and testing.

The Qt for Python wheels 🛞

Building the project has many steps, and with that some extra levels of complexity that can affect the final packages, so one needs to put extra effort on check that everything is properly working.

If you are familiar with building and packaging Python projects, we are using two approaches

• setup.py approach relying on 'build', 'install' and 'bdist_wheel', and
• setup.py approach with setup.cfg and pyproject.toml to use the build module.

why two? because we don't want to deprecate the first one before getting used to the second approach, by not using distutils (due to its EOL in 3.12) and trying to see where the Python community is evolving to improve the packaging story. Learn more about the current recommendations here. In case you are wondering, the wheels we provide are based on the second approach.

For one project/wheel you call the setup.py build once, but in our case we have an ad-hoc recursive process that runs three times for our three projects: shiboken6, shiboken6-generator, and PySide6. This is good for development processes, but it does include some little issues, like not being able to run pip install . inside the pyside-setup repository, but we are trying to improve our processes further on to solve that in future releases.

Final thoughts 🐍

It is not the first time we have an issue on a new release, and we are certain it will not be the last, but we really did not want to keep the Python community hanging and unable to use Python 3.11, after all it was one of the highlights in our 6.4 release.

Python 3.11 is a really important release, if you are not familiar with the changes, please read the release notes, from which we can highlight that it can be up to 60% faster than 3.10 for some python tests, fine grained error locations in trace-backs, exception groups, and more!

We know that in a couple of weeks we will have 6.4.1, but in the meantime, please enjoy 6.4.0.1 and help us by reporting any issue you find 🐛

Would you like to learn about another internal process? let us know in the comments so we can blog about it in the future!