There is git-annex-remote-ora and all supporting functionality in datalad core. Unfortunately it operates only at the full file extraction level ATM, and we would need some indexed access within a file. If I get it right , here is a relevant discussion https://stackoverflow.com/questions/4457997/indexing-random-access-to-7zip-7z-archives and also https://github.com/vasi/lzopfs (Random access to compressed files with a FUSE filesystem) might be of interest.
Not yet sure if worthwhile.
Then there would be a need in fsspec layer to provide fetching of corresponding file from ORA.
Ultimate goal of a kind: In combination with #34 (bare) and #35 (plain directory, not super dataset) could provide a quite useful setup for HPCs where there is virtually no inodes consumption -- FUSE layer would take care about providing access to heavy trees etc
There is
git-annex-remote-oraand all supporting functionality in datalad core. Unfortunately it operates only at the full file extraction level ATM, and we would need some indexed access within a file. If I get it right , here is a relevant discussion https://stackoverflow.com/questions/4457997/indexing-random-access-to-7zip-7z-archives and also https://github.com/vasi/lzopfs (Random access to compressed files with a FUSE filesystem) might be of interest.Not yet sure if worthwhile.
Then there would be a need in fsspec layer to provide fetching of corresponding file from ORA.
Ultimate goal of a kind: In combination with #34 (bare) and #35 (plain directory, not super dataset) could provide a quite useful setup for HPCs where there is virtually no inodes consumption -- FUSE layer would take care about providing access to heavy trees etc