Change irods mountpoint to mirror actual logical path of file

[theferrit32]

For example if I have file /prodzone/home/theferrit32/file.csv if a user wants to use it in a workflow in a workspace, currently it is made available at /renci/irods/home/theferrit32/file.csv. This is not very intuitive, and though it could be explained in documentation, it could also be changed to be more intuitive.

One possibility is to set the mountpoint to be the zone name so the file would be available in the container at the unix file path /prodzone/home/theferrit32/file.csv. This is the most intuitive, however could create problems if someone names an iRODS zone lib and then that volume gets mounted over top of /lib in the container. It's not that this sort of edge case is dangerous to the platform, but it is unlikely that the workflow would be able to execute.

Another possibility is to have an unused path prepended, to arrive at something like:
/irods/prodzone/home/theferrit32/file.csv

And another is to use a custom protocol type, which might also be used in other places at some point in the near future:
irods:///prodzone/home/theferrit32/file.csv
This one would require CWL extensions to define a new type and handler for this, because it is not just a File type.

[mr-c]

In CWL,File.location can be any URI known to the engine, not just file:///, http://, https://.
So the following is a valid CWL File object, no need to create a new type:

class: File
location: irods:///prodzone/home/theferrit32/file.csv

[theferrit32]

@mr-c Interesting, thanks for the clarification on that. As part of the work on BDBag fetching, we plan to have a python library with a straightforward way to authenticate and fetch irods resources, so that could be plugged into the cwltool/toil libraries.

It looks like I could subclass the Fetcher class to provide handling of irods:// URIs, and pass in a fetcher_constructor to use that class instead of the DefaultFetcher.

constructor:
https://github.com/common-workflow-language/cwltool/blob/83541ecc7c1b8497e0cdbec021a5449f14678da4/cwltool/main.py#L331

fetcher handling, to be extended in subclass:
https://github.com/common-workflow-language/schema_salad/blob/18294270c20c640c7aba7a34d2efe84718f41008/schema_salad/ref_resolver.py#L138

Does this seem like the appropriate way to add this functionality, or do you have any additional pointers on the best way to extend the resource fetching?

[mr-c]

@theferrit32 You are welcome.

Yes, that is how Arvados does it: https://github.com/curoverse/arvados/blob/606c29cdbb9012ee99043a2da9f28b2cd302e5e4/sdk/cwl/arvados_cwl/__init__.py#L859

https://github.com/curoverse/arvados/blob/606c29cdbb9012ee99043a2da9f28b2cd302e5e4/sdk/cwl/arvados_cwl/fsaccess.py#L191