Skip to content

Commit dd24476

Browse files
valeriupredoivaleriupredoi
authored
Merge branch 'main' into anon_bucket
2 parents 4df1b33 + 685cf6f commit dd24476

17 files changed

Lines changed: 138 additions & 72 deletions

‎.github/dependabot.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# To get started with Dependabot version updates, you'll need to specify which
2+
# package ecosystems to update and where the package manifests are located.
3+
# Please see the documentation for all configuration options:
4+
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
5+
6+
version: 2
7+
updates:
8+
# Enable version updates for GitHub Actions
9+
- package-ecosystem: "github-actions"
10+
# Workflow files stored in the default location of `.github/workflows`
11+
# You don't need to specify `/.github/workflows` for `directory`. You can use `directory: "/"`.
12+
directory: "/"
13+
schedule:
14+
interval: "weekly"
15+
cooldown:
16+
default-days: 7
17+
labels:
18+
- dependencies
19+
- github_actions
20+
- automatedPR

‎.github/workflows/build-and-deploy-on-pypi.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,11 @@ jobs:
1616
# IMPORTANT: this permission is mandatory for trusted publishing on PyPI or Test PyPI
1717
id-token: write
1818
steps:
19-
- uses: actions/checkout@v5
19+
- uses: actions/checkout@v6
2020
with:
2121
fetch-depth: 0
2222
- name: Set up Python 3.14
23-
uses: actions/setup-python@v4
23+
uses: actions/setup-python@v6
2424
with:
2525
python-version: "3.14"
2626
- name: Install pep517

‎.github/workflows/create-condalock-file.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
name: Create and verify conda lock file for latest Python
2121
runs-on: 'ubuntu-latest'
2222
steps:
23-
- uses: actions/checkout@v5
23+
- uses: actions/checkout@v6
2424
with:
2525
fetch-depth: 0
2626
- uses: conda-incubator/setup-miniconda@v3
@@ -64,7 +64,7 @@ jobs:
6464
# Automated PR
6565
# see https://github.com/marketplace/actions/create-pull-request
6666
- name: Create Automated PR if conda lock file has changed
67-
uses: peter-evans/create-pull-request@v3
67+
uses: peter-evans/create-pull-request@v8
6868
with:
6969
token: ${{ secrets.GITHUB_TOKEN }}
7070
commit-message: Updating condalock file

‎.github/workflows/install-from-condalock-file.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
fail-fast: false
2626
name: Linux Python ${{ matrix.python-version }}
2727
steps:
28-
- uses: actions/checkout@v5
28+
- uses: actions/checkout@v6
2929
with:
3030
fetch-depth: 0
3131
- uses: conda-incubator/setup-miniconda@v3

‎.github/workflows/run-s3-test-push.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
fail-fast: false
2020
name: Linux Python ${{ matrix.python-version }}
2121
steps:
22-
- uses: actions/checkout@v5
22+
- uses: actions/checkout@v6
2323
with:
2424
fetch-depth: 0
2525
- name: Export proxy

‎.github/workflows/run-test-push.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
fail-fast: false
1818
name: Linux Python ${{ matrix.python-version }}
1919
steps:
20-
- uses: actions/checkout@v5
20+
- uses: actions/checkout@v6
2121
with:
2222
fetch-depth: 0
2323
- uses: conda-incubator/setup-miniconda@v3
@@ -33,4 +33,4 @@ jobs:
3333
- run: conda list
3434
- run: flake8 --exclude tests,doc --max-line-length 120 --ignore F405,F401
3535
- run: pytest -n 2 -m "not slow" --junitxml=report-1.xml
36-
- uses: codecov/codecov-action@v5
36+
- uses: codecov/codecov-action@v6

‎.github/workflows/run-tests-condaforge.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
fail-fast: false
2222
name: Linux Python ${{ matrix.python-version }}
2323
steps:
24-
- uses: actions/checkout@v5
24+
- uses: actions/checkout@v6
2525
with:
2626
fetch-depth: 0
2727
- uses: conda-incubator/setup-miniconda@v3
@@ -54,7 +54,7 @@ jobs:
5454
fail-fast: false
5555
name: OSX Python ${{ matrix.python-version }}
5656
steps:
57-
- uses: actions/checkout@v5
57+
- uses: actions/checkout@v6
5858
with:
5959
fetch-depth: 0
6060
- uses: conda-incubator/setup-miniconda@v3

‎.github/workflows/run-tests.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
fail-fast: false
2222
name: Linux Python ${{ matrix.python-version }}
2323
steps:
24-
- uses: actions/checkout@v5
24+
- uses: actions/checkout@v6
2525
with:
2626
fetch-depth: 0
2727
- uses: conda-incubator/setup-miniconda@v3
@@ -46,7 +46,7 @@ jobs:
4646
fail-fast: false
4747
name: OSX Python ${{ matrix.python-version }}
4848
steps:
49-
- uses: actions/checkout@v5
49+
- uses: actions/checkout@v6
5050
with:
5151
fetch-depth: 0
5252
- uses: conda-incubator/setup-miniconda@v3

‎.github/workflows/test_s3_minio.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
fail-fast: false
2424
name: Linux Python ${{ matrix.python-version }}
2525
steps:
26-
- uses: actions/checkout@v5
26+
- uses: actions/checkout@v6
2727
with:
2828
fetch-depth: 0
2929
- name: Export proxy

‎activestorage/active.py‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,8 @@ def __init__(self,
190190
interface_type: str = None,
191191
max_threads: int = 100,
192192
storage_options: dict = None,
193-
active_storage_url: str = None) -> None:
193+
active_storage_url: str = None,
194+
option_disable_chunk_cache: bool = False) -> None:
194195
"""
195196
Instantiate with a NetCDF4 dataset URI and the variable of interest within that file.
196197
(We need the variable, because we need variable specific metadata from within that
@@ -199,6 +200,7 @@ def __init__(self,
199200
200201
:param storage_options: s3fs.S3FileSystem options
201202
:param active_storage_url: Reductionist server URL
203+
:param option_disable_chunk_cache: flag to disable chunk caching
202204
"""
203205
self.ds = None
204206
input_variable = False
@@ -256,6 +258,9 @@ def __init__(self,
256258
self.storage_options = storage_options
257259
self.active_storage_url = active_storage_url
258260

261+
# turn off chunk caching
262+
self.option_disable_chunk_cache = option_disable_chunk_cache
263+
259264
# basic check on file
260265
if not input_variable:
261266
if not os.path.isfile(self.uri) and not self.interface_type:
@@ -661,6 +666,9 @@ def _process_chunk(self,
661666
# Axes over which to apply a reduction
662667
axis = self._axis
663668

669+
# turn off chunk caching
670+
chunk_caching = self.option_disable_chunk_cache
671+
664672
if self.interface_type == 's3' and self._version == 1:
665673
tmp, count = reduce_opens3_chunk(ds._fh,
666674
offset,
@@ -703,7 +711,8 @@ def _process_chunk(self,
703711
ds._order,
704712
chunk_selection,
705713
axis,
706-
operation=self._method)
714+
operation=self._method,
715+
option_disable_chunk_cache=chunk_caching,)
707716
else:
708717
if self.storage_options.get("anon", None) is True:
709718
bucket = os.path.dirname(parsed_url.path)
@@ -723,7 +732,8 @@ def _process_chunk(self,
723732
ds._order,
724733
chunk_selection,
725734
axis,
726-
operation=self._method)
735+
operation=self._method,
736+
option_disable_chunk_cache=chunk_caching,)
727737
elif self.interface_type == "https" and self._version == 2:
728738
tmp, count = reductionist.reduce_chunk(session,
729739
self.active_storage_url,
@@ -739,7 +749,8 @@ def _process_chunk(self,
739749
chunk_selection,
740750
axis,
741751
operation=self._method,
742-
interface_type="https")
752+
interface_type="https",
753+
option_disable_chunk_cache=chunk_caching,)
743754

744755
elif self.interface_type == 'ActivePosix' and self.version == 2:
745756
# This is where the DDN Fuse and Infinia wrappers go

0 commit comments

Comments
 (0)