Compiling ncclx from source without conda

[castuardo]

Hello,

I am trying to compile ncclx from source without using conda. I used the command:

CONDA_PREFIX=<path>/ncclx ./build_ncclx.sh 

When installing deps, I keep failing on Thrift with something like

[113/533] Linking CXX executable bin/thrift1
FAILED: bin/thrift1 
: && /usr/bin/c++  -Wl,--allow-shlib-undefined thrift/compiler/CMakeFiles/thrift1.dir/main.cc.o thrift/compiler/CMakeFiles/thrift1.dir/compiler.cc.o -o bin/thrift1  -Wl,-rpath,<path>/ncclx/lib::::::::::::::::::::::::::::::::::::::  lib/libcompiler_ast.a  lib/libcompiler_lib.a  -Wl,--whole-archive  lib/libcompiler_generators.a  -Wl,--no-whole-archive  lib/libcompiler_lib.a  lib/libcompiler.a  lib/libcompiler_ast.a  lib/libwhisker.a  lib/libcompiler_base.a  <path>/ncclx/lib/libfolly.a  <path>/ncclx/lib/libboost_context.so.1.82.0  <path>/ncclx/lib/libboost_filesystem.so.1.82.0  <path>/ncclx/lib/libboost_atomic.so.1.82.0  <path>/ncclx/lib/libboost_program_options.so.1.82.0  <path>/ncclx/lib/libboost_regex.so.1.82.0  <path>/ncclx/lib/libboost_thread.so.1.82.0  <path>/ncclx/lib/libdouble-conversion.a  <path>/ncclx/lib/libgflags.so.2.2.2  <path>/ncclx/lib/libglog.a  <path>/ncclx/lib/libevent.a  <path>/ncclx/lib/libz.a  <path>/ncclx/lib/libssl.a  <path>/ncclx/lib/libcrypto.a  -ldl  <path>/ncclx/lib/libzstd.a  <path>/ncclx/lib/libsodium.a  -ldl  <path>/ncclx/lib/libzstd.a  <path>/ncclx/lib/libsodium.a  lib/libcommon.a  <path>/ncclx/lib/libfmt.so.11.2.0 && :
/usr/bin/ld: <path>/ncclx/lib/libglog.a(logging.cc.o): in function `__static_initialization_and_destruction_0(int, int)':
logging.cc:(.text+0x6658): undefined reference to `google::FlagRegisterer::FlagRegisterer<bool>(char const*, char const*, char const*, bool*, bool*)'
/usr/bin/ld: logging.cc:(.text+0x671c): undefined reference to `google::FlagRegisterer::FlagRegisterer<bool>(char const*, char const*, char const*, bool*, bool*)'
/usr/bin/ld: logging.cc:(.text+0x67cd): undefined reference to `google::FlagRegisterer::FlagRegisterer<bool>(char const*, char const*, char const*, bool*, bool*)'
/usr/bin/ld: logging.cc:(.text+0x687e): undefined reference to `google::FlagRegisterer::FlagRegisterer<bool>(char const*, char const*, char const*, bool*, bool*)'
/usr/bin/ld: logging.cc:(.text+0x68bc): undefined reference to `google::FlagRegisterer::FlagRegisterer<int>(char const*, char const*, char const*, int*, int*)'
/usr/bin/ld: logging.cc:(.text+0x6970): undefined reference to `google::FlagRegisterer::FlagRegisterer<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)'

Is this a known issue, and if so, is there any fix?

[d4l3k]

This isn't a known issue. I expect somehow it's pulling in a different version of gflags? Take a look at the CI scripts for how to build this from scratch. It may be you have an incompatible version of gflags in your conda/system env

[castuardo]

Ill give it a shot and update, thank you so much.