Is it possible to relax requires-python version on pyodk?

1. What is the issue? Please be detailed.

  • pyodk requires a minimum of Python 3.12.
  • Python 3.12 is only about a year old.
  • Installing pyodk in a project with Python < 3.12 fails, else the version locked by a dependency solver is 0.3.0 (about a year and a half old).
  • For context, Python 3.10 still has another two years until end of life.

2. What steps can we take to reproduce this issue?

  • Attempt to install pyodk in a Python 3.11 project.

3. Question

  • Does pyodk use functionality from Python 3.12 that make it backward incompatible?
  • Would it be possible to specify requires-python to 3.11, or even 3.10, giving a broader range of support?
  • I would understand pinning to such a recent version of Python if this was something like a CLI for end users. But as it's a developer facing API wrapper, then it will probably be installed as a dependency of other projects (that are not necessarily on Python 3.12 yet).

With the limited resources available, selecting a single supported python version makes it more feasible to deliver a good experience for users and add features they need, by avoiding the maintenance burden of including and testing more supported environments.

It's good to hear that experienced developers find pyodk useful as part of larger projects. The pyodk library is also targeted at users that are writing simpler utility scripts (like the examples in the docs folder), which may be their first scripts in Python (or ever) e.g. from the readme:

This library aims to make common data analysis and workflow automation tasks as simple as possible

Is there a reason why your project cannot use python 3.12?

1 Like

Thanks for the explanation!

I understand the concern of limited resources and wishing to reduce maintenance burden. That's actually one of the answers to your last question about why we can't upgrade to 3.12 immediately!

But the key reason would be that I see no need to restrict our package (the one that installs pyodk) to 3.12, and thus limit the potential users. Many people are stuck on Python <3.12 for various reasons, one being that other installed packages may not support 3.12 so dependency locking fails.

I see no reason why pyodk would not work on 3.10, in fact I override the min version in my fork and use the code in a 3.10 environment :slight_smile: Unless there is specific Python 3.11 or 3.12 syntax in a project, then it should be fine to allow for a minimum version of 3.10. Syntax differences between 3.10-3.12 are minimal, mostly around typing.

End user software can upgrade is quickly as possible, while dependency packages should ideally use minimum versions to be flexible. The user can decide if they want to run the package in 3.10, 3.11, or 3.12.

1 Like

Perhaps an option could be to say that Python 3.12 is recommended, and testing is only carried out on Python 3.12.

But allow the user to install on 3.10 onwards, with a note that it's at their own risk and not officially supported?

1 Like

I just had an experience related to this. I wanted to run a script I had tested locally on another environment. I couldn't get pyodk to upgrade from v0.3.0. Even though I'd read this thread a few days ago it wasn't until I tried to force the install of the latest version that I realized the issue is that the environment I'm deploying to has Python 3.8.10.

I think that sounds like a good compromise. @Lindsay_Stevens_Au what do you think?

2 Likes

To further complicate things, the most recent version of pyodk also has three dependencies pinned:

The == pinning makes pyodk unusable / uninstallable in projects that have shared dependencies (pydantic, requests, toml) that specify different versions. For pydantic, pydantic>=2.6.4,<3 would also be more ideal.

I know semver can't be 100% relied upon, so I understand the desire to reduce risk. Both requests and pydantic are well established and very professional projects, so I would find it hard to imagine them breaking semver. toml is more of a one man show, but in reality how much does the toml spec and functionality change here?

In summary, I'm not necessarily requesting that these pins be made more flexible (although that would be lovely!). I guess it would just be good to have clarity on what the target for pyodk is:

  • A helpful wrapper to be used in simple scripts (I would imagine this is most usage currently).
  • Both the above, plus an installable API for others projects to utilise ODK Central functionality easily (currently the project is not really usable for this due to strict python and dependency version pinning).

If the latter is desirable, I'm more than happy to help in any way I can to make this a reality. Of course, the former is also fine too, but it would be good to clarify so I will continue work on my own Central API wrapper instead :smile:

(also, I know this project has low resources allocated to it, so I apologise for causing a fuss!)

@LN From the packaging user guide tutorial on requires-python:

requires-python gives the versions of Python supported by your project. An installer like pip will look back through older versions of packages until it finds one that has a matching Python version.

And metadata specifications:

This field specifies the Python version(s) that the distribution is compatible with. Installation tools may look at this when picking which version of a project to install.

This package metadata is a statement on compatibility. If I installed a package that said it supported a particular python version and it didn't work, I would probably contact the maintainer to fix the package metadata or the bug. It can only be known that pyodk works with a specific python version by testing it, and adding that support would increase maintenance overhead. I'm not comfortable with the alternative of misusing requires-python and implicitly recommending that users put untested code into production. While there might not (or might - it's not tested) currently be actual syntax/compatibility issues with earlier python versions, by supporting them it means that pyodk cannot (rather than "happens to not") use language features that are newer than the oldest supported version.

Thanks Lindsay!

I completely agree with the sentiment that we don't want increased maintenance burden for you :smiley:

I am a bit confused though: it seemed from your original reply that pyodk is intended mostly for simple scripting, but your latest reply mentions concerns for pyodk in production (indicating it is intended to be used within production software, for which I think the current config is rather restrictive).

Either way, as I said I don't want to make your life difficult, so if you feel it's too much additional work to support Python <3.12 then I understand!