When the HDF5 ROS3 driver is invoked, only the region, access key ID, and secret key are currently passed. However, to access NASA Earthdata S3 buckets, a session token is also required. While the HDF5 ROS3 driver does support a session token, it is not currently being utilized.
It would be beneficial to:
-
Add support for passing a session token to the ROS3 driver.
-
Parse all potential sources of credentials (e.g., environment variables and AWS config files) to determine the most appropriate set of credentials for accessing multiple buckets.
-
Add support for S3 URIs such as:
s3://ob-cumulus-prod-public/PACE_OCI.20240411T000331.L1B.V3.nc
I’ve created a development branch to address this issue:
👉 https://github.com/alexandervladsemenov/netcdf-c/tree/temp_credentials
The code has been tested and is functional. However, I believe the current mechanism for parsing URLs/URIs in netCDF could benefit from a broader redesign. I'd like to get input from the netCDF development team before proceeding further.
I’ll submit a draft pull request to facilitate feedback and discussion.
When the HDF5 ROS3 driver is invoked, only the region, access key ID, and secret key are currently passed. However, to access NASA Earthdata S3 buckets, a session token is also required. While the HDF5 ROS3 driver does support a session token, it is not currently being utilized.
It would be beneficial to:
Add support for passing a session token to the ROS3 driver.
Parse all potential sources of credentials (e.g., environment variables and AWS config files) to determine the most appropriate set of credentials for accessing multiple buckets.
Add support for S3 URIs such as:
s3://ob-cumulus-prod-public/PACE_OCI.20240411T000331.L1B.V3.nc
I’ve created a development branch to address this issue:
👉 https://github.com/alexandervladsemenov/netcdf-c/tree/temp_credentials
The code has been tested and is functional. However, I believe the current mechanism for parsing URLs/URIs in netCDF could benefit from a broader redesign. I'd like to get input from the netCDF development team before proceeding further.
I’ll submit a draft pull request to facilitate feedback and discussion.