Skip to content

Comments

Port to HighFive for kealib 2.0#82

Open
gillins wants to merge 94 commits intomasterfrom
v2_highfive
Open

Port to HighFive for kealib 2.0#82
gillins wants to merge 94 commits intomasterfrom
v2_highfive

Conversation

@gillins
Copy link
Member

@gillins gillins commented Nov 14, 2025

Very WIP, just wanting to check tests on different OS's

@gillins
Copy link
Member Author

gillins commented Dec 9, 2025

Merry Christmas @petebunting! I think this PR is ready for your review....

I think this makes the HighFive port fully functional (although I haven't tested it with rsgislib yet). It does appear to work fine on OSX with the conda toolchain (I've added a github action to test).

There were a few bits were I needed to drop back to C:

  1. File property lists. HighFive does not support all the properties like fill value etc.
  2. Reading imagery where only part of the image is within the requested area. HighFive doesn't seem to support this situation where the buffer is a different size to what is being read.
  3. Neighbours. While HighFive supports variable length strings, it doesn't support variable length ints etc.

I have implemented tests for every piece of funcionality that kealib has. These tests are split into a writer and reader for each data type. The backport_testsuite branch has this test suite backported against the existing kealib. This means you can run the reader from the existing kealib with files created by the writer from the new one and vice versa. You can also compare the files using h5dump and h5dump -p (small differences in sizes and offsets appear normal). The differences in the RAT fill value are due to the old kealib not being able to export a kea_att_file to another band - had to write the test suite differently here.

The bundled GDAL driver now passes the GDAL KEA testsuite. I have updated the minimum GDAL version for the driver to 3.6 and managed to clear up some #ifdefs as a result.

GDAL 3.12 has added functionality for storing datetime and WKB's as columns in the RAT. I've added functions for this, but they raise an error. Because I already have the functions the ABI won't change when they are implemented. I'm hoping to put out a point release with this functionality after we release kealib 2.0.

There are a few changes that may impact users of kealib:

  1. You now get a HighFive::File* from openKeaH5RW etc. I'll submit a PR for GDAL (this is the only change for GDAL to compile). I do wonder if we should be returning our own opaque type that wraps HighFive::File* so if we ever change how kealib works again the users won't have to change their code. What do you think?
  2. The RAT class hierachy is a bit different. Previously, KEAAttributeTable was a virtual base class. If can now be instantiated and has common implementations as much as possible. I can't imagine anyone is using these classes directly anyway.
  3. The old cmake "libkea" config has finally been removed. Users should be using the new "Kealib" config instead. We've had both for a while.
  4. Minimum GDAL for bundled plugin has been bumped as above

Anyway, let me know what you think. I'm hoping to get a release out around Easter. I'm going to spend the time between now and then on general testing, completing the doxygen and general tidy up. Keen to hear if there is anything else that needs changing.

@gillins gillins requested a review from petebunting December 9, 2025 23:00
@gillins gillins marked this pull request as ready for review December 9, 2025 23:00
@gillins
Copy link
Member Author

gillins commented Jan 16, 2026

@bhjolly @petescarth @neilflood @tonykgill @danclewley @t-hackwood @rouault @petebunting @eco-nz @shepjd

Hi All, just letting you know that the new HighFive based kealib is ready for some initial Alpha testing.

Background: The C++ bindings to hdf5 (which kealib currently uses) are to be retired at some stage. Additionally, there appears to be a problem on OSX that causes kealib to crash. So this version uses HighFive which appears to be well maintained and provides a simpler interface onto hdf.

If you have time to test this (not in production), I'd appreciate hearing about any problems. The new library should read files created with older kealib and vice versa.

Currently gdal does not know how to build against this version of kealib (PR forthcoming) but you should be able to install the standalone GDAL KEA plugin (LIBKEA_WITH_GDAL=ON in your cmake command, and set GDAL_PLUGIN_PATH appropriately). Instructions below for conda on Linux/OSX (on Windows you'll also need to install the MS compiler).

conda create -n highfive cmake cxx-compiler hdf5 gdal highfive make
conda activate highfive
conda remove libgdal-kea

git clone -b v2_highfive https://github.com/ubarsc/kealib.git kealib_highfive
cd kealib_highfive
mkdir build
cd build
cmake -D CMAKE_INSTALL_PREFIX=$CONDA_PREFIX -D CMAKE_PREFIX_PATH=$CONDA_PREFIX -D LIBKEA_WITH_GDAL=ON ..
make
ctest -V
make install

You should now have a working kealib install with GDAL support etc. Let me know if you have any problems.

@gillins
Copy link
Member Author

gillins commented Jan 22, 2026

@t-hackwood is reporting intermittent "Can't find BandTest1 in image metadata" type errors in the tests on OSX. They go away when the address sanitizer is turn on... Has anyone else seen anything similar? All ok on the github build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants