Unified Snap by snadrus · Pull Request #806 · filecoin-project/curio

snadrus · 2025-11-17T23:29:41Z

This addition to the fast snap mainline modifies supraseal:

No more objcopy: An interface glues the filereader with the NVMe driver.
Faster build using parallelism
AMD64v3 becomes the default, as Curio has had for a while (2013-2015 era introduction).
Added a fallback for CPUs without SHA-NI (v3 but not v4) to be able to use Supraseal.

This should make fastsnap the only path needed for Curio.

mbx2 is the crazy SHA for Zen: supraseal/sha/sha_mbx_wrapper.cpp

Cases
old libfilecoin-ffi snap:

non-Linux: mbx2 fallback
Linux Snap without CUDA
Linux before AMD64v4

Use supraseal fastsnap:

Linux with Cuda with AMD64v4

Use supraseal PC1:

Linux + CUDA + Zen only. TODO!!!!!!!: Fix mbx2 to run only in this case.

DEBs to cover all cases:

OPENGL
CUDA
ZEN

Note

Unifies SupraSeal snap path with CPU/GPU fallback, adds a curio batch setup SPDK command, refactors/builds the SupraSeal library, and overhauls CI/Docker to Ubuntu 24.04 with GCC 12 and CUDA.

SupraSeal core/library:
- Unifies PC2 via templated readers (NVMe and file) and removes objcopy/symbol hacks.
- Parallelizes build (Poseidon constants, objects); adjusts NVCC flags; builds libsupraseal.a (no tool binaries in CI).
- Adds CUDA-only TreeR path with CPU/GPU feature detection; fallback to filecoin-ffi when unsupported.
- Fixes memory delete (delete[]) in c1/challenge.hpp.
CLI/runtime:
- Adds curio batch setup to auto-configure SPDK (hugepages, NVMe binding).
- Auto SPDK setup and hugepage checks in batch task; supports DISABLE_SPDK_SETUP env.
Go bindings (lib/supraffi, ffidirect):
- Splits NVMe vs generic CUDA paths; adds CPU feature and CUDA probing; TreeR fallback in FFI wrapper.
Build/Makefile:
- Makes SupraSeal a default Linux build dep; sets GOAMD64=v3; removes legacy batch targets.
CI/Docker:
- Migrates to ubuntu-24.04, replaces custom Go action, sets GCC 12 via alternatives.
- Installs CUDA 12+, configures stubs and env; recursive submodules; caches proof params.
- Adds dedicated SupraSeal library build job/artifact and local test Dockerfile.
Docs:
- Updates install deps (adds libgmp-dev, libconfig++-dev), switches build instructions to make curio/sptool.
- Documents curio batch setup and new env var in EN/ZN SupraSeal guides.

^{Written by Cursor Bugbot for commit 23c6b6a. This will update automatically on new commits. Configure here.}

magik6k · 2025-11-24T10:33:20Z

This is cool, but I really don't think it's all that useful, all somewhat recent CPUs have sha extensions, and without those I don't think the performance will be useful even for testing

We got pushback when a make Curio's minimum AMD64v4 earlier this year, so we lowered it to V3 which lacks sha-ni.

Is V3 == -march=zenver2 or are those kinda different?

snadrus · 2025-12-01T21:13:29Z

+	return hasAMD64v4
+}
+
+func detectAMD64v4() {


@magik6k how should I detect "Zen-ish" for SupraPC1 ?

Checking CPU support for SHA-NI should be good enough

snadrus · 2025-12-12T23:52:53Z

Cannot proceed until the self-hosted runners get to Ubuntu 24.04 (currently 20.04 which was also referred to as "ubuntu-latest")

magik6k · 2025-12-12T16:00:24Z

Is V3 == -march=zenver2 or are those kinda different?

magik6k · 2025-12-13T23:29:02Z

+// Include file reader implementation with renamed class to avoid conflict
+#define streaming_node_reader_t streaming_node_reader_files_t
+#include "../c1/streaming_node_reader_files.hpp"
+#undef streaming_node_reader_t


This seems really extremely cursed

cursor

This PR is being reviewed by Cursor Bugbot

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

magik6k · 2025-12-15T07:39:13Z

Merging into andyfastsnap so that there's just one branch to work on

cursor · 2025-12-15T07:45:38Z

+        if [ -n "${LD_LIBRARY_PATH}" ]; then
+          LD_LIB_PATH="$LD_LIB_PATH:${LD_LIBRARY_PATH}"
+          LIB_PATH="$LIB_PATH:${LIBRARY_PATH}"
+        fi


Bug: Mismatched environment variable check causes incorrect library paths

The condition on line 165 checks if LD_LIBRARY_PATH is non-empty, but line 167 unconditionally appends ${LIBRARY_PATH} (a different environment variable) to LIB_PATH. These are two distinct environment variables (LD_LIBRARY_PATH for runtime linking, LIBRARY_PATH for compile-time linking). If LD_LIBRARY_PATH is set but LIBRARY_PATH is unset, this appends an empty value causing a trailing colon in the path. Conversely, if LIBRARY_PATH is set but LD_LIBRARY_PATH is not, the value never gets appended. Each variable needs its own separate check.

* supra: file tree_r ffi * wire up treerfile to go * supra: using the correct node reader in file tree_r ffi * tree_r links * redefs, it works * missing test-supra cmd * fast snap encode * encode-snap test * wire up fast-snap * make gen * encode works * fix: yield in PSClientPoll correctly * faster webui history * pspoll: Don't timeout the context * fix EpochPretty ddos * more robust dealdata * robusthttp metrics * enable is done this way round * pshare: prio snap * encode bind to data * make gen * piecepark node dl limit * only park pieces locally when supra P2 isn't running * fix commp canaccept * fix build * fix commp canaccept storage query * aaaargh * prevent dupe piece park task * expose piecepark min free pct * Do snap falloc * Cache storage info in localstorage.Local calls for a bit * MoveStorage / UpgradeStore param tweaks * snap key no read * snap: Parallel movestorage * snap mertics, ui fix * missing snap metrics * fix snap batch backoff * somewhat handle snap msg fails on immutable deadline * fix snap retry query params * storage: Copy over snap vproof * treeR file deadline * slower retry movestorage * ipni: Handle orphan tasks * ipni: Accept orphans * f05 bmgr * f05 bmgr fixes * fix f05 bm build * make gen * missing fixes postrebase * fix: proofshare: Fix client payment chain bootstrap * feat: test post cli: Vanilla Proof test command * supra: live duration-so-far metric * webui: Content page * fix: Make kubo retrievals work (#724) * fix(retrieval): handle pathed IPFS retrieval style, add basic retrieval tests (#720) * ipni: Announce addresses with port * update lotus * make gen --------- Co-authored-by: Rod Vagg <rod@vagg.org> * add more caches * more metrics, less log spam * http method in req stats * http metric for in-flight requests * remotebs: return correct err on not-found * limit parallel retrievals * separate head req limiter * fix race condition that resulted in ignoring the last bit of cached data. * merge iffy * gen * lint and tests * old linter making mistakes * linty * fixes * oops * agent review fixes * cuda13 capable ffi * supra: allow passing march flag * Unified Snap (#806) * build * 2 * 3 * build gotchas * right shape * requirements include cuda * get build fixed * cuda for everyone * add golang for ci * stop deleting my stuff * try with gcc flag * auto-nvme-setup opt-out * ci build fix attempt * fix outdated base image * gofmt, flag- no avx512bf16 * try cuda setup * do like working job * try without cuda * try cude in install-deps * cuda info in github env * make spdk happy * deps * gcc12 * ubuntu-latest is not 2404 but 2004 * ci: missing deps, fix ubuntus * supra: rm mbx wrapper, fix poseidon constants wait, native default arch * review fixes --------- Co-authored-by: Łukasz Magiera <magik6k@gmail.com> * version and dep * dl v1 * missing market miners onchange initial setup * test supra system-info command * supra: add missing pc2_hash_files template instantiations * supra: correct -march in pc2.cu to make blst not segfault * cache supraseal * fix: storage: Make declare usably fast * sectormetadata: Recheck partition presence * debug expmgr * expmgr: Never try to extend dead sectors * use our runners * Trying to fix CI * fixing CI * indexing: gracefully handle no-unsealed-copy pieces * solve CI lint errors * CI cleanups * CI: export regex * test_env_vars * cgo fix * paux in fallback flow * gen * cuda probe without nvcc * env to disable snap supra * fix generic crosscompile, explicit native target * gcc 13 for supra * supra docs * address review * fix itests --------- Co-authored-by: Łukasz Magiera <magik6k@gmail.com> Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com> Co-authored-by: Rod Vagg <rod@vagg.org>

snadrus added 5 commits November 17, 2025 17:04

build

f0adbf5

2

80b13d1

3

a75978b

build gotchas

ce61748

right shape

c1ff11a

magik6k reviewed Nov 24, 2025

View reviewed changes

snadrus commented Dec 1, 2025

View reviewed changes

snadrus added 20 commits December 1, 2025 17:06

requirements include cuda

92dcb17

get build fixed

3510737

cuda for everyone

6aa9b2c

add golang for ci

b10b53e

stop deleting my stuff

532d805

try with gcc flag

8a658b4

auto-nvme-setup opt-out

3d9d894

ci build fix attempt

f9bbec6

fix outdated base image

1276638

gofmt, flag- no avx512bf16

2c1a5a3

Merge branch 'andyfastsnap' into andyfastsnap-unified

3098150

try cuda setup

ce03125

do like working job

b15f3b6

try without cuda

6382970

try cude in install-deps

9da8cd8

cuda info in github env

362424b

make spdk happy

dfdca24

deps

c9a9cc4

gcc12

21c671b

ubuntu-latest is not 2404 but 2004

2ee3810

magik6k force-pushed the andyfastsnap-unified branch 2 times, most recently from d77abff to 27984aa Compare December 13, 2025 14:35

magik6k force-pushed the andyfastsnap-unified branch from 27984aa to ec248bc Compare December 13, 2025 14:38

ci: missing deps, fix ubuntus

ca7f08e

magik6k force-pushed the andyfastsnap-unified branch 2 times, most recently from ce76fab to ca7f08e Compare December 13, 2025 18:26

magik6k marked this pull request as ready for review December 13, 2025 23:25

magik6k requested a review from a team as a code owner December 13, 2025 23:25

magik6k reviewed Dec 13, 2025

View reviewed changes

cursor Bot reviewed Dec 13, 2025

View reviewed changes

Comment thread lib/ffiselect/ffidirect/ffi-direct.go

supra: rm mbx wrapper, fix poseidon constants wait, native default arch

0508662

cursor Bot reviewed Dec 13, 2025

View reviewed changes

Comment thread lib/supraffi/cuda_linux.go Outdated

review fixes

23c6b6a

magik6k merged commit 1c7fa06 into andyfastsnap Dec 15, 2025
6 of 14 checks passed

magik6k deleted the andyfastsnap-unified branch December 15, 2025 07:39

cursor Bot reviewed Dec 15, 2025

View reviewed changes

Uh oh!

Conversation

snadrus commented Nov 17, 2025 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

magik6k Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

snadrus Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

magik6k Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

snadrus Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

magik6k Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

snadrus commented Dec 12, 2025

Uh oh!

magik6k Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

magik6k Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

This PR is being reviewed by Cursor Bugbot

Uh oh!

Uh oh!

Uh oh!

magik6k commented Dec 15, 2025

Uh oh!

Uh oh!

cursor Bot Dec 15, 2025

Choose a reason for hiding this comment

Bug: Mismatched environment variable check causes incorrect library paths

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

snadrus commented Nov 17, 2025 •

edited by cursor Bot

Loading