Unable to rechunk a view of a 2-d sliced AnnData with a dask array

[joshua-gould]

Please make sure these conditions are met

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of anndata.
• (optional) I have confirmed this bug exists on the main branch of anndata.

Report

Code:

import anndata
import dask.array as da
import numpy as np

x = da.random.random((10, 10))
d = anndata.AnnData(x)
x2 = x[[0, 1, 4]][..., [0, 1, 4]]
x2 = x2.rechunk('auto')

d2 = d[[0, 1, 4], [0, 1, 4]]
np.testing.assert_equal(d2.X.compute(), x2.compute())
d2.X = d2.X.rechunk(
    'auto')  # Incorrect shape ((3, 1)) of integer indices for dimension with size 10


Traceback:

```pytb
  File "lib/python3.12/site-packages/anndata/_core/anndata.py", line 678, in X
    self._adata_ref._X[oidx, vidx] = value
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "lib/python3.12/site-packages/dask/array/core.py", line 1968, in __setitem__
    dsk = setitem_array(out, self, key, value)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/site-packages/dask/array/slicing.py", line 1712, in setitem_array
    indices, implied_shape, reverse, implied_shape_positions = parse_assignment_indices(
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/site-packages/dask/array/slicing.py", line 1395, in parse_assignment_indices
    raise IndexError(
IndexError: Incorrect shape ((3, 1)) of integer indices for dimension with size 10

Versions

| Dependency                    | Version     |
| ----------------------------- | ----------- |
| setuptools                    | 80.9.0      |
| sphinxcontrib-jquery          | 4.1         |
| sphinxcontrib-htmlhelp        | 2.1.0       |
| cloudpickle                   | 3.1.2       |
| cycler                        | 0.12.1      |
| python-dateutil               | 2.9.0.post0 |
| pillow                        | 11.3.0      |
| Pygments                      | 2.19.2      |
| omegaconf                     | 2.3.0       |
| pyparsing                     | 3.3.1       |
| argcomplete                   | 3.6.3       |
| sphinxcontrib-qthelp          | 2.0.0       |
| sphinxcontrib-jsmath          | 1.0.1       |
| pure_eval                     | 0.2.3       |
| six                           | 1.17.0      |
| session-info2                 | 0.3         |
| executing                     | 2.2.1       |
| pytz                          | 2025.2      |
| packaging                     | 25.0        |
| ipython                       | 9.9.0       |
| parso                         | 0.8.5       |
| asttokens                     | 3.0.1       |
| matplotlib                    | 3.10.8      |
| prompt_toolkit                | 3.0.52      |
| decorator                     | 5.2.1       |
| defusedxml                    | 0.7.1       |
| pandas                        | 2.3.3       |
| pyarrow                       | 23.0.0      |
| wcwidth                       | 0.2.14      |
| sphinxcontrib-devhelp         | 2.0.0       |
| sphinxcontrib-applehelp       | 2.0.0       |
| numpy                         | 1.26.4      |
| traitlets                     | 5.14.3      |
| jedi                          | 0.19.2      |
| kiwisolver                    | 1.4.9       |
| stack-data                    | 0.6.3       |
| sphinxcontrib-serializinghtml | 2.0.0       |
| Component | Info                                                                              |
| --------- | --------------------------------------------------------------------------------- |
| Python    | 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:34:53) [Clang 19.1.7 ] |
| OS        | macOS-26.2-arm64-arm-64bit                                                        |
| Updated   | 2026-02-05 15:17

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
Please add a comment if you want to keep the issue open. Thank you for your contributions!

[joshua-gould]

Reopening

[ilan-gold]

FWIW @joshua-gould I think the big blocker here is propagating updated views back to parents. So anndata -> view of anndata -> update element in view of anndata at the moment causes the view to be instantiated as an "actual" anndata object.

That is what the dask array view class is for and why it is so broken (presumably as this issue is evidence of) I didn't look into why this was broken TBH because I started commenting on #2329. In short, I see two solutions here:

1. Somehow encode a reference to the anndata object for updates without actually doing whatever it is that is breaking things here
2. Patch-fix the underlying problem here.

[joshua-gould]

What's the status of removing the view mechanism in anndata? Thanks.

[ilan-gold]

I think the point 1. above is the biggest blocker. Even if an element is copy-on-write, that doesn't really solve our issue wholesale. The other part is making sure that the parent object is updated under some reasonable circumstances. Right now, our custom classes provide ways of doing this by _SetItemMixin or similar that does updated under view_update which allows instantiating the view of the AnnData as an actual.

Without this ability, I'm not sure what something like adata[subset].obs["new_col"] = my_new_data is supposed to do.

So we either figure out a way to do this that doesn't involve subclassing (note, you also need to handle the "detached" case obs_subset = adata[subset].obs followed by update(obs_subset)), or we need some form of "mixed views" which would likely boil down to a key-value store of updated/instantiated objects. So adata[subset].obs["new_col"] = my_new_data would then bring only obs into memory for the subset (and in this case be immediately discarded but adata_subset.obs["new_col"] = my_new_data would hold on to the newly instantiated subset of obs while remaining a view for the other things like X, obsm etc.).

[joshua-gould]

I don't think anndata needs copy on write. It should use the behavior of the underlying X array which could be numpy, cupy etc. For pandas 3, copy on write is already the default behavior. In contrast to anndata, xarray does not implement copy on write, so users don't run into issues with computing/assigning on subsets and I find easy to understand. Thanks.

import anndata
import numpy as np
import pandas as pd
import xarray as xr

obs = pd.DataFrame(
    {"test1": [1, 2, 1, 2, 1, 2, 1, 2, 1, 2], "test2": [1, 2, 1, 2, 1, 2, 1, 2, 1, 2]})
# https://pandas.pydata.org/docs/user_guide/copy_on_write.html
obs_subset = obs[obs['test1'] == 2]
obs_subset['test3'] = 3
assert "test3" not in obs.columns
coords = {}
for c in obs.columns:
    coords[c] = ("obs", obs[c])

# xarray
xarray_data = xr.DataArray(np.zeros((10, 10)), dims=["obs", "var"],
                           coords=coords)

xarray_data1 = xarray_data[slice(0, 2)]  # view
assert xarray_data1.data.base is not None  # assert view
xarray_data1[0, 0] = 1  # modified original array
assert xarray_data1[0, 0] == 1

xarray_data2 = xarray_data[[0, 1]]  # copy
assert xarray_data2.data.base is None  # assert copy
xarray_data2[0, 0] = 2
assert xarray_data1[0, 0] == 1  # does not modify original array

# anndata
anndata_data = anndata.AnnData(np.zeros((10, 10)), obs=obs)
anndata_data1 = anndata_data[slice(0, 2)]
anndata_data1.X[0, 0] = 1  # does not modify original array
assert anndata_data.X[0, 0] == 0
anndata_data2 = anndata_data[[0, 1]]
anndata_data2.X[0, 0] = 2  # does not modify original array
assert anndata_data.X[0, 0] == 0

[ilan-gold]

anndata_data1.X[0, 0] = 1 # does not modify original array

My point is that this behavior instantiates anndata_data1 in-memory and it is no longer a view. Did you see a warning when doing this operation? And being able to instantiate this view into memory is a crucial guarantee we make at the moment.

[joshua-gould]

I do get a warning. However, I don't think having copy on write is crucial for anndata to keep as almost all subsets users create (e.g. subsetting to highly variables genes) can not be expressed using offsets and strides and therefore are already copies. What about modifying https://github.com/scverse/anndata/blob/main/src/anndata/_core/index.py#L106 to return an index array for mutable array types so that backwards compatibility is maintained and no copy on write behavior is needed? Thanks.