introduce a version.txt for ament_python

[chrisdonlan]

Hey, onboarding to ros2. This avent_python template looks really good/standard for operational use from my perspective as a drone/embedded/hardware/IoT fleet CI/CD and C2 guy. It looks almost identical to the model templates I set up for employers actually, which has motivated me to make a comment :-).

Saw that the team here is still pretty small, and I'll be adapting this template with version.txt, thought I'd propose it here or at least start the conversation.

I really like the use of version.txt next to setup.py which I typically recommend referencing in the module's __init__.py file dynamically (along with other changes so that setup.py is standard and copy-able from one project to the next). I pulled this concept from java libraries, but it is something I've seen in production in IoT industry. I make use of it in my infra, and it's useful for allowing CI/CD + shell routines to easily hook into changes in the version.txt file in addition to Cxx sidecar libraries and other add ins that accumulate IRL.

Further, it is very easy to increment a semver in a version.txt rather than either pyproject.toml, setup.py, or module_name/__init__.py via normal git and CICD workflows (written in shell or abbreviated shell environments in alpine linux, for example), whereas jumping into any of those other file types can require extra work and packages in the build container to support whatever one liner we come up with (using perl/awk/sed, jq, etc.). It's also pretty fragile when not using version.txt and supporting many teams, as these python packaging approaches tend to be trendy rather than good, changing every 18 months (the versioning methods wind up being as numerous as the projects we support).

[christophebedard]

What template are you referring to?

[chrisdonlan]

The python template, ament_cmake_python project structure: I am proposing adding a version.txt to contain the semantic version. I'll be adapting things that way on my side, and while y'all aren't using the version.txt, your ament_cmake_python project structure looks almost identical to what I've been creating and using for my employers...

... so I figure we come from a common development family. Figured I'd mention that is what I am doing in my hardware in the loop setups and IoT deployments, maybe it is useful.

[chrisdonlan]

.
├── LICENSE
├── my_py_package
│   ├── __init__.py
│   ├── my_py_node.py
│   └── py.typed
├── package.xml
├── resource
│   └── my_py_package
├── setup.cfg
├── setup.py
└── test
    ├── test_copyright.py
    ├── test_flake8.py
    ├── test_mypy.py
    ├── test_pep257.py
    └── test_xmllint.py

-- this is what I am getting after running ros2 pkg create --build-type ament_python --lisence Apache-2.0 --node-name my_py_node my_py_package

What I'll be adding to make this fit in with my general CI/CD workflows and whatnot:

├── LICENSE
├── my_py_package
│   ├── __init__.py
│   ├── my_py_node.py
│   └── py.typed
├── package.xml
├── resource
│   └── my_py_package
├── setup.cfg
├── setup.py
├── test
│   ├── test_copyright.py
│   ├── test_flake8.py
│   ├── test_mypy.py
│   ├── test_pep257.py
│   └── test_xmllint.py
└── version.txt  <---------------------------------- this file

Which is the file I'll hinge CI/CD (.gitlab etc), bash scripts and git hooks on. I'll source it dynamically in python and set an environment variable from cmake/GNU make when integrating Cxx and Python in the same projects, use it for docker image tags and so on.

Figure it might be worth thinking about.

Apologies if I found the wrong issue board, continuing to get used to ROS2 and family of repos.

[christophebedard]

Thanks for clarifying. What you're talking about is the ros2pkg package in the ros2/ros2cli repo: https://github.com/ros2/ros2cli/tree/rolling/ros2pkg/ros2pkg/resource/ament_python. Unfortunately, it seems like I can't move this issue across GitHub orgs. (And, yes, I know all these separate repos, and even separate GitHub orgs, is a bit confusing!)

As to your contribution/feature request, I would say it doesn't align with how ROS packages are maintained, at least as things stand*. The canonical place to define a ROS package version is in the pacakge.xml file, see REP 149: https://reps.openrobotics.org/rep-0149/#version. For C++ packages, that's the only place it's defined (thanks to ament_package() for ament_cmake packages). For Python packages, due to how Python works, it also needs to be defined in the setup.py file. However, we are actually looking to get rid of that, see this (unfortunately) old issue: ros2/ros2#1132.

The reason is that, when you create a new release for a package, you need to bump the version number. Having to bump the version number in multiple locations is annoying and error-prone. We have tooling to bump the version number, but we still wouldn't really recommend adding another location for the version number. Does that make sense?

*Now, if Python is moving towards defining a Python module's version in a version.txt file instead of the setup.py file (I found related discussions by Googling), then we should definitely discuss it. We're looking to better align with newer standards-based Python packaging. For instance, see this effort to align colcon (ROS 2's build tool) with newer Python packaging practices: https://discourse.openrobotics.org/t/call-for-testing-standards-based-python-packaging-with-colcon/32008 + https://github.com/colcon/colcon-python-project/tree/devel. This is working towards getting rid of setup.py, which, IIUC, aligns with your goal. What do you think?

tl;dr I'm not an expert in this area, but your request should probably be done in conjunction with ROS Python package packaging modernization (e.g., colcon-python-project), and not on its own.

[christophebedard]

Also, if you just want a tool to detect ROS packages under a given directory and extract their versions (by reading their package.xml file) to integrate into CI systems, we have catkin_pkg: https://github.com/ros-infrastructure/catkin_pkg. See: find_packages() + Package.version.