Skip to content

Memory heap usage of /vsimem/ files is incorrect #217

@mmomtchev

Description

@mmomtchev

Tracking the amount of memory allocated in /vismem/ files is currently incorrect as there is no way track the changes to the amount of memory used after the ownership has been transferred to GDAL.

If a file is created using only GDAL tools (such as gdal.translate) or if a Node.js Buffer is transferred to GDAL using vsimem.copy, GDAL will be free to allocate and deallocate memory. Transferring this file back to Node.js using vsimem.release will signal to the GC that it releases the amount of memory held at this point of time - which may be different from the initial amount of memory.

The only good solution is to implement VSIMalloc/VSIFree hooking. GDAL's documentation contains vague references to a hooking API, but it does not seem to exist.

As the latest versions of Node.js 23 as well as the recent Node.js 24 feature heap size consistency checks, if GDAL extends a file that is later vsimem.released, this triggers a new assertion check:

# Check failed: static_cast<int64_t>(amount_before) >= -delta.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions