Error writing `recarray` with all `nan` values

[lazappi]

Please make sure these conditions are met

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of anndata.
• (optional) I have confirmed this bug exists on the main branch of anndata.

Report

I get errors when writing an AnnData that contains a recarray with all nan values (coming from scanpy.tl.filter_rank_genes_groups()) to either H5AD or Zarr.

H5AD error

TypeError: Can't implicitly convert non-string objects to strings
Error raised while writing key 'names' of <class 'h5py._hl.group.Group'> to /uns/rank_genes_groups_filtered

Zarr error

AttributeError: 'float' object has no attribute 'encode'
Error raised while writing key 'names' of <class 'zarr.core.group.Group'> to /uns/rank_genes_groups_filtered

Problematic data

adata.uns["rank_genes_groups_filtered"]

{'params': {'groupby': 'group',
  'reference': 'rest',
  'method': 't-test',
  'use_raw': False,
  'layer': None,
  'corr_method': 'benjamini-hochberg'},
 'names': rec.array([(nan, nan), (nan, nan), (nan, nan), (nan, nan), (nan, nan),
            (nan, nan), (nan, nan), (nan, nan), (nan, nan), (nan, nan),
            (nan, nan)],
           dtype=[('0', 'O'), ('1', 'O')]),
 'scores': rec.array([( 2.2136757 ,  1.4461874 ), ( 1.0155576 ,  1.2298588 ),
            ( 0.5710249 ,  0.7113681 ), ( 0.5113449 ,  0.61132497),
            ( 0.28787297,  0.00831682), ( 0.13881163, -0.13881163),
            (-0.00831682, -0.28787297), (-0.61132497, -0.5113449 ),
            (-0.7113681 , -0.5710249 ), (-1.2298588 , -1.0155576 ),
            (-1.4461874 , -2.2136757 )],
           dtype=[('0', '<f4'), ('1', '<f4')]),
 'pvals': rec.array([(0.05859625, 0.18716785), (0.34069169, 0.26948629),
            (0.58832636, 0.49708198), (0.62324728, 0.55870866),
            (0.7815499 , 0.99359452), (0.89481534, 0.89481534),
            (0.99359452, 0.7815499 ), (0.55870866, 0.62324728),
            (0.49708198, 0.58832636), (0.26948629, 0.34069169),
            (0.18716785, 0.05859625)],
           dtype=[('0', '<f8'), ('1', '<f8')]),
 'pvals_adj': rec.array([(0.64455878, 0.856965  ), (0.856965  , 0.856965  ),
            (0.856965  , 0.856965  ), (0.856965  , 0.856965  ),
            (0.95522765, 0.99359452), (0.98429687, 0.98429687),
            (0.99359452, 0.95522765), (0.856965  , 0.856965  ),
            (0.856965  , 0.856965  ), (0.856965  , 0.856965  ),
            (0.856965  , 0.64455878)],
           dtype=[('0', '<f8'), ('1', '<f8')]),
 'logfoldchanges': rec.array([(-6.5878111e-01,  1.1797867e+00),
            ( 2.0365384e+00,  1.2380860e+00),
            ( 3.5793650e-01,  6.9241780e-01),
            ( 6.5031129e-01,  5.1760316e-01),
            ( 3.1437758e-01, -7.8160426e-04),
            (-3.7294958e-02,  3.7294958e-02),
            ( 7.8160426e-04, -3.1437758e-01),
            (-5.1760316e-01, -6.5031129e-01),
            (-6.9241780e-01, -3.5793650e-01),
            (-1.2380860e+00, -2.0365384e+00),
            (-1.1797867e+00,  6.5878111e-01)],
           dtype=[('0', '<f4'), ('1', '<f4')])}

This is related to several existing issues but I thought it was worth opening a new one to:

• Collect them in one place (see below)
• I think I have a nice reproducible example
• I don't think the Zarr error has been reported before

Reproducible example

# /// script
# requires-python = ">=3.12"
# dependencies = [
#   "anndata@git+https://github.com/scverse/anndata.git",
# ]
# ///

import scanpy as sc
import pandas as pd

adata = sc.datasets.blobs(n_centers = 1, n_observations = 10)
adata.obs["group"] = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1]
adata.obs["group"] = adata.obs["group"].astype("category")

sc.tl.rank_genes_groups(adata, groupby="group")
sc.tl.filter_rank_genes_groups(adata, groupby="group")

Tracebacks

adata.write_h5ad()

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[8], line 1
----> 1 adata.write_h5ad("test.h5ad")

    [... skipping hidden 1 frame]

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_core/anndata.py:1922, in AnnData.write_h5ad(self, filename, convert_strings_to_categoricals, compression, compression_opts, as_dense)
   1919 if filename is None:
   1920     filename = self.filename
-> 1922 write_h5ad(
   1923     Path(filename),
   1924     self,
   1925     convert_strings_to_categoricals=convert_strings_to_categoricals,
   1926     compression=compression,
   1927     compression_opts=compression_opts,
   1928     as_dense=as_dense,
   1929 )
   1930 # Only reset the filename if the AnnData object now points to a complete new copy
   1931 if self.isbacked and not self.is_view:

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:349, in no_write_dataset_2d.<locals>.raise_error_if_dataset_2d_present(store, adata, *args, **kwargs)
    342     msg = (
    343         "Writing AnnData objects with a Dataset2D not supported yet. "
    344         "Please use `ds.to_memory` to bring the dataset into memory. "
    345         "Note that if you have generated this object by concatenating several `AnnData` objects"
    346         "the original types may be lost."
    347     )
    348     raise NotImplementedError(msg)
--> 349 return write(store, adata, *args, **kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/h5ad.py:105, in write_h5ad(filepath, adata, as_dense, convert_strings_to_categoricals, dataset_kwargs, **kwargs)
    103 write_elem(f, "varp", dict(adata.varp), dataset_kwargs=dataset_kwargs)
    104 write_elem(f, "layers", dict(adata.layers), dataset_kwargs=dataset_kwargs)
--> 105 write_elem(f, "uns", dict(adata.uns), dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:518, in write_elem(store, k, elem, dataset_kwargs)
    494 def write_elem(
    495     store: GroupStorageType,
    496     k: str,
   (...)    499     dataset_kwargs: Mapping[str, Any] = MappingProxyType({}),
    500 ) -> None:
    501     """
    502     Write an element to a storage group using anndata encoding.
    503 
   (...)    516         E.g. for zarr this would be `chunks`, `compressor`.
    517     """
--> 518     Writer(_REGISTRY).write_elem(store, k, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:389, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    386 write_func = self.find_write_func(dest_type, elem, modifiers)
    388 if self.callback is None:
--> 389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
    390 return self.callback(
    391     write_func,
    392     store,
   (...)    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:405, in write_mapping(f, k, v, _writer, dataset_kwargs)
    403 g = f.require_group(k)
    404 for sub_k, sub_v in v.items():
--> 405     _writer.write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:389, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    386 write_func = self.find_write_func(dest_type, elem, modifiers)
    388 if self.callback is None:
--> 389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
    390 return self.callback(
    391     write_func,
    392     store,
   (...)    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:405, in write_mapping(f, k, v, _writer, dataset_kwargs)
    403 g = f.require_group(k)
    404 for sub_k, sub_v in v.items():
--> 405     _writer.write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:389, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    386 write_func = self.find_write_func(dest_type, elem, modifiers)
    388 if self.callback is None:
--> 389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
    390 return self.callback(
    391     write_func,
    392     store,
   (...)    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:686, in write_recarray(f, k, elem, _writer, dataset_kwargs)
    676 @_REGISTRY.register_write(H5Group, (np.ndarray, "V"), IOSpec("rec-array", "0.2.0"))
    677 @_REGISTRY.register_write(H5Group, np.recarray, IOSpec("rec-array", "0.2.0"))
    678 def write_recarray(
   (...)    684     dataset_kwargs: Mapping[str, Any] = MappingProxyType({}),
    685 ):
--> 686     f.create_dataset(k, data=_to_hdf5_vlen_strings(elem), **dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/h5py/_hl/group.py:193, in Group.create_dataset(self, name, shape, dtype, data, **kwds)
    190         parent_path, name = name.rsplit(b'/', 1)
    191         group = self.require_group(parent_path)
--> 193 dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
    194 dset = dataset.Dataset(dsid)
    195 return dset

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/h5py/_hl/dataset.py:180, in make_new_dset(parent, shape, dtype, data, name, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times, external, track_order, dcpl, dapl, efile_prefix, virtual_prefix, allow_unknown_filter, rdcc_nslots, rdcc_nbytes, rdcc_w0, fill_time)
    177 dset_id = h5d.create(parent.id, name, tid, sid, dcpl=dcpl, dapl=dapl)
    179 if (data is not None) and (not isinstance(data, Empty)):
--> 180     dset_id.write(h5s.ALL, h5s.ALL, data)
    182 return dset_id

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File h5py/h5d.pyx:305, in h5py.h5d.DatasetID.write()

File h5py/_proxy.pyx:153, in h5py._proxy.dset_rw()

File h5py/_conv.pyx:439, in h5py._conv.str2vlen()

File h5py/_conv.pyx:109, in h5py._conv.generic_converter()

File h5py/_conv.pyx:245, in h5py._conv.conv_str2vlen()

TypeError: Can't implicitly convert non-string objects to strings
Error raised while writing key 'names' of <class 'h5py._hl.group.Group'> to /uns/rank_genes_groups_filtered

adata.write_zarr()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[9], line 1
----> 1 adata.write_zarr("test.zarr")

    [... skipping hidden 1 frame]

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_core/anndata.py:2004, in AnnData.write_zarr(self, store, chunks, convert_strings_to_categoricals)
   1999     msg = (
   2000         "Passing `write_zarr(adata, chunks=True)` is no longer supported. "
   2001         "Please pass `write_zarr(adata)` instead."
   2002     )
   2003     raise ValueError(msg)
-> 2004 write_zarr(
   2005     store,
   2006     self,
   2007     chunks=chunks,
   2008     convert_strings_to_categoricals=convert_strings_to_categoricals,
   2009 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:349, in no_write_dataset_2d.<locals>.raise_error_if_dataset_2d_present(store, adata, *args, **kwargs)
    342     msg = (
    343         "Writing AnnData objects with a Dataset2D not supported yet. "
    344         "Please use `ds.to_memory` to bring the dataset into memory. "
    345         "Note that if you have generated this object by concatenating several `AnnData` objects"
    346         "the original types may be lost."
    347     )
    348     raise NotImplementedError(msg)
--> 349 return write(store, adata, *args, **kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/zarr.py:59, in write_zarr(store, adata, chunks, convert_strings_to_categoricals, **ds_kwargs)
     56         dataset_kwargs = dict(dataset_kwargs, chunks=chunks)
     57     write_func(store, elem_name, elem, dataset_kwargs=dataset_kwargs)
---> 59 write_dispatched(f, "/", adata, callback=callback, dataset_kwargs=ds_kwargs)
     60 if is_zarr_v2():
     61     zarr.convenience.consolidate_metadata(f.store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/experimental/_dispatch_io.py:74, in write_dispatched(store, key, elem, callback, dataset_kwargs)
     70 from anndata._io.specs import _REGISTRY, Writer
     72 writer = Writer(_REGISTRY, callback=callback)
---> 74 writer.write_elem(store, key, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:390, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    388 if self.callback is None:
    389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
--> 390 return self.callback(
    391     write_func,
    392     store,
    393     k,
    394     elem,
    395     dataset_kwargs=dataset_kwargs,
    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/zarr.py:57, in write_zarr.<locals>.callback(write_func, store, elem_name, elem, dataset_kwargs, iospec)
     51 if (
     52     chunks is not None
     53     and not isinstance(elem, sparse.spmatrix)
     54     and elem_name.lstrip("/") == "X"
     55 ):
     56     dataset_kwargs = dict(dataset_kwargs, chunks=chunks)
---> 57 write_func(store, elem_name, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:305, in write_anndata(f, k, adata, _writer, dataset_kwargs)
    303 _writer.write_elem(g, "varp", dict(adata.varp), dataset_kwargs=dataset_kwargs)
    304 _writer.write_elem(g, "layers", dict(adata.layers), dataset_kwargs=dataset_kwargs)
--> 305 _writer.write_elem(g, "uns", dict(adata.uns), dataset_kwargs=dataset_kwargs)
    306 _writer.write_elem(g, "raw", adata.raw, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:390, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    388 if self.callback is None:
    389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
--> 390 return self.callback(
    391     write_func,
    392     store,
    393     k,
    394     elem,
    395     dataset_kwargs=dataset_kwargs,
    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/zarr.py:57, in write_zarr.<locals>.callback(write_func, store, elem_name, elem, dataset_kwargs, iospec)
     51 if (
     52     chunks is not None
     53     and not isinstance(elem, sparse.spmatrix)
     54     and elem_name.lstrip("/") == "X"
     55 ):
     56     dataset_kwargs = dict(dataset_kwargs, chunks=chunks)
---> 57 write_func(store, elem_name, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:405, in write_mapping(f, k, v, _writer, dataset_kwargs)
    403 g = f.require_group(k)
    404 for sub_k, sub_v in v.items():
--> 405     _writer.write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:390, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    388 if self.callback is None:
    389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
--> 390 return self.callback(
    391     write_func,
    392     store,
    393     k,
    394     elem,
    395     dataset_kwargs=dataset_kwargs,
    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/zarr.py:57, in write_zarr.<locals>.callback(write_func, store, elem_name, elem, dataset_kwargs, iospec)
     51 if (
     52     chunks is not None
     53     and not isinstance(elem, sparse.spmatrix)
     54     and elem_name.lstrip("/") == "X"
     55 ):
     56     dataset_kwargs = dict(dataset_kwargs, chunks=chunks)
---> 57 write_func(store, elem_name, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:405, in write_mapping(f, k, v, _writer, dataset_kwargs)
    403 g = f.require_group(k)
    404 for sub_k, sub_v in v.items():
--> 405     _writer.write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/utils.py:272, in report_write_key_on_error.<locals>.func_wrapper(*args, **kwargs)
    270     raise ValueError(msg)
    271 try:
--> 272     return func(*args, **kwargs)
    273 except Exception as e:
    274     path = _get_display_path(store)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:390, in Writer.write_elem(self, store, k, elem, dataset_kwargs, modifiers)
    388 if self.callback is None:
    389     return write_func(store, k, elem, dataset_kwargs=dataset_kwargs)
--> 390 return self.callback(
    391     write_func,
    392     store,
    393     k,
    394     elem,
    395     dataset_kwargs=dataset_kwargs,
    396     iospec=self.registry.get_spec(elem),
    397 )

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/zarr.py:57, in write_zarr.<locals>.callback(write_func, store, elem_name, elem, dataset_kwargs, iospec)
     51 if (
     52     chunks is not None
     53     and not isinstance(elem, sparse.spmatrix)
     54     and elem_name.lstrip("/") == "X"
     55 ):
     56     dataset_kwargs = dict(dataset_kwargs, chunks=chunks)
---> 57 write_func(store, elem_name, elem, dataset_kwargs=dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/registry.py:77, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     75 @wraps(func)
     76 def wrapper(g: GroupStorageType, k: str, *args, **kwargs):
---> 77     result = func(g, k, *args, **kwargs)
     78     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     79     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/_io/specs/methods.py:701, in write_recarray_zarr(f, k, elem, _writer, dataset_kwargs)
    689 @_REGISTRY.register_write(ZarrGroup, (np.ndarray, "V"), IOSpec("rec-array", "0.2.0"))
    690 @_REGISTRY.register_write(ZarrGroup, np.recarray, IOSpec("rec-array", "0.2.0"))
    691 def write_recarray_zarr(
   (...)    697     dataset_kwargs: Mapping[str, Any] = MappingProxyType({}),
    698 ):
    699     from anndata.compat import _to_fixed_length_strings
--> 701     elem = _to_fixed_length_strings(elem)
    702     if is_zarr_v2():
    703         f.create_dataset(k, data=elem, shape=elem.shape, **dataset_kwargs)

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/compat/__init__.py:409, in _to_fixed_length_strings(value)
    406 for dt_name, (dt_type, dt_offset) in value.dtype.fields.items():
    407     if dt_type.kind == "O":
    408         #  Assuming the objects are str
--> 409         size = max(len(x.encode()) for x in value.getfield("O", dt_offset))
    410         new_dtype.append((dt_name, ("U", size)))
    411     else:

File ~/Documents/Code/data-intuitive/pfizer-spatial/.pixi/envs/default/lib/python3.12/site-packages/anndata/compat/__init__.py:409, in <genexpr>(.0)
    406 for dt_name, (dt_type, dt_offset) in value.dtype.fields.items():
    407     if dt_type.kind == "O":
    408         #  Assuming the objects are str
--> 409         size = max(len(x.encode()) for x in value.getfield("O", dt_offset))
    410         new_dtype.append((dt_name, ("U", size)))
    411     else:

AttributeError: 'float' object has no attribute 'encode'
Error raised while writing key 'names' of <class 'zarr.core.group.Group'> to /uns/rank_genes_groups_filtered

Related issues

anndata

• Can't implicitly convert non-string objects to strings. Bug about tl.filter_rank_genes_groups and tl.rank_genes_groups #1141
• (Semi-)automatic conversion of nullable columns to the appropriate pandas arrays #1068
• IO for nullable string arrays #1558

scanpy

• TypeError: Can't implicitly convert non-string objects to strings  scanpy#2321
• Can't implicitly convert non-string objects to strings. Bug about tl.filter_rank_genes_groups and tl.rank_genes_groups scanpy#2666

Versions

Package	Version
anndata	0.12.9

Dependency	Version
wcwidth	0.5.3
executing	2.2.1
Jinja2	3.1.6
typing_extensions	4.15.0
PyYAML	6.0.3
parso	0.8.5
donfig	0.8.1.post1
scanpy	1.12
asttokens	3.0.1
lz4	4.4.5
charset-normalizer	3.4.4
h5py	3.15.1
MarkupSafe	3.0.3
pytz	2025.2
pure_eval	0.2.3
scipy	1.16.3
packaging	26.0
pandas	2.3.3
numcodecs	0.16.5
zarr	3.1.5
tblib	3.2.2
traitlets	5.14.3
jedi	0.19.2
fsspec	2026.1.0
natsort	8.4.0
dask	2024.11.2
psutil	7.2.2
cytoolz	1.1.0
python-dateutil	2.9.0.post0
backports.zstd	1.3.0
stack_data	0.6.3
fast-array-utils	1.3.1
threadpoolctl	3.6.0
google-crc32c	1.8.0
decorator	5.2.1
numpy	2.3.5
toolz	1.1.0
cloudpickle	3.1.2
pyarrow	20.0.0
session-info2	0.4
msgpack	1.1.2
prompt_toolkit	3.0.52
Pygments	2.19.2
six	1.17.0
ipython	9.10.0
legacy-api-wrap	1.5
setuptools	80.10.2
xarray	2026.1.0

Component	Info
Python	3.12.12
OS	Linux-6.17.0-14-generic-x86_64-with-glibc2.39
Updated	2026-03-11 15:39

[ilan-gold]

Hmmm @lazappi is names expected to be all nan?

[lazappi]

In this example, yes. In general usage, hopefully not because some genes pass the thresholds but it's fairly easy to make happen depending on what groups you test, what the data looks like etc. (hence the existing issues where other people have had the same problem). It came up for me because I was using a tiny test dataset but I'm had to work around it with real data before.

I think anndata should handle the writing better but maybe scanpy should also avoid creating these results in the first place.

[ilan-gold]

I think anndata should handle the writing better but maybe scanpy should also avoid creating these results in the first place.

Ok this seems reasonable. I will need to look a bit into serializing nan into object. It may require stringify-ing (since these aren't really numbers anyway). I will also need to understand this example a bit as to why these are nan and maybe if we should error out.

[lazappi]

The filter_rank_genes_groups() function tries to identify "markers" using thresholds on log fold changes and percentage of expressing cells inside/outside each group. The way it stores those results is to duplicate the rank_genes_groups() results but with the var name set to nan if that feature is not a "marker". In this example, no vars pass the thresholds so all the names are nan which I think causes the issue (but maybe not entirely). I'm not sure why this way of storing the results was chosen and it's not very intuitive or convenient to me but that's a separate problem.

[ilan-gold]

@flying-sheep do you have any thoughts here on what we should do? These are real nans and silently writing them might not be ideal...

[JiwaniZakir]

The root cause is that the names field in the recarray has object dtype ('O') — normally used for gene name strings — but scanpy.tl.filter_rank_genes_groups() fills filtered-out entries with float('nan') instead of a string sentinel. The anndata I/O layer (likely in anndata/_io/specs/types.py or wherever structured arrays are encoded) assumes all object-dtype fields in a recarray are string-typed and passes values directly to h5py/zarr string encoding, which then chokes on the bare nan float. The fix would be to either coerce object-dtype recarray fields to strings before writing (e.g., converting nan → "nan") or detect the mixed-type case and raise a cleaner error. On the scanpy side, using "" or np.nan cast to str as the fill value would also sidestep the issue entirely.

[flying-sheep]

@JiwaniZakir please stop posting LLM summaries, especially since this issue (and the scanpy issue where you also did that) already have good summaries.

[JiwaniZakir]

The root cause is that recarray string fields containing nan (float) values aren't being coerced before the write path attempts to call .encode() on them. A minimal fix is to convert those nan entries to empty strings (or a sentinel like "") before writing: arr[np.isnan(arr.astype(float))] = "" won't work cleanly on structured arrays, but iterating over dtype fields and replacing nans in object/string columns should do it. If you're on anndata ≥ 0.11, setting anndata.settings.allow_write_nullable_strings = True may also sidestep the issue by using a nullable string representation that tolerates missing values.