Skip to content

Add meson build system version 2#2734

Open
oscarbenjamin wants to merge 104 commits into
flintlib:mainfrom
oscarbenjamin:pr_meson2
Open

Add meson build system version 2#2734
oscarbenjamin wants to merge 104 commits into
flintlib:mainfrom
oscarbenjamin:pr_meson2

Conversation

@oscarbenjamin

Copy link
Copy Markdown
Contributor

This PR adds a meson build configuration to FLINT and is a follow on from gh-1910 taking a slightly different approach. That PR added a meson build configuration as an optional bolt on with a script meson_boostrap.py that copies/generates meson build files into the source tree. The reason for doing it that way is because I envisaged that it would be optional/experimental at least for some time.

After discussion with @albinahlback the idea here is to make a PR that fully replaces the autotools build with meson so that we can see how that looks. There are a few things not yet complete that would be needed for that but the goal for now is to get to the point where all CI jobs are switched to using meson and all are passing.

The approach here adds files like meson.build directly in the source tree but also has a Python script called update_meson.py that generates a meson.build file in every directory under src. These files will explicitly list all .c files that need to be compiled so the intention is that they would be automatically generated.

If using meson as the main build system it would be best to commit all of these files to git even though they are generated files. This is so that every commit has a fully statically specified build configuration. I have not yet committed them to git in this PR in the interest of keeping the diff small (the diff would be something like 10000 lines when first added). Also I have not removed the existing autotools files for the same reason.

In the first push here I have just added a CI job to test the meson build on Linux x86-64 with gcc. The other CI jobs still use the autotools build but I will push subsequent commits to change them as well.

@lgoettgens

Copy link
Copy Markdown
Contributor

Noting that individual test program becomes build/src/acb/test/main instead of build/src/acb/test/main.

I don't see any difference between the two (@fredrik-johansson )

2. Run python3 update_meson.py. This is only temporarily needed because the generated meson.build files have not been checked into git. The idea is that eventually this step would only be needed when adding or removing a .c file to keep the meson.build files in sync with the sources.

I see this as a likely root of future issues. Would it be worth it to add a light-weight CI job that just checks out the repository, runs python3 update_meson.py and fails if git status --porcelain has non-empty output? We have something similar in https://github.com/oscar-system/Oscar.jl/blob/c97ff243c8c346ebc713a78a057a8638bad3bf10/.github/workflows/BibtoolCI.yml#L27-L53

@lgoettgens

Copy link
Copy Markdown
Contributor

While trying to compile from this branch (commit 8ba9d64) locally, I got some warnings. See the thread below:

$ uname -a
Linux E15 6.18.32-1-lts #1 SMP PREEMPT_DYNAMIC Sun, 17 May 2026 18:36:29 +0000 x86_64 GNU/Linux
$ cat /etc/os-release 
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo
$ meson -v
1.11.1
$ ninja --version 
1.13.2
$ python3 update_meson.py 
Copying /home/lgoe/code/flint/meson.build to /home/lgoe/code/flint/meson.build
Copying /home/lgoe/code/flint/meson.options to /home/lgoe/code/flint/meson.options
Copying /home/lgoe/code/flint/include/meson.build to /home/lgoe/code/flint/include/meson.build
Copying /home/lgoe/code/flint/include/flint/meson.build to /home/lgoe/code/flint/include/flint/meson.build
Copying /home/lgoe/code/flint/config.m4.in to /home/lgoe/code/flint/config.m4.in
Writing /home/lgoe/code/flint/src/meson.build
Making meson.build files in all modules
$ meson setup build       
The Meson build system
Version: 1.11.1
Source dir: /home/lgoe/code/flint
Build dir: /home/lgoe/code/flint/build
Build type: native build
Project name: FLINT
Project version: 3.6.0-dev
C compiler for the host machine: /usr/bin/ccache cc (gcc 16.1.1 "cc (GCC) 16.1.1 20260430")
C linker for the host machine: cc ld.bfd 2.46.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Library m found: YES
Found pkg-config: YES (/usr/bin/pkg-config) 2.5.1
Run-time dependency gmp found: YES 6.3.0
Run-time dependency mpfr found: YES 4.2.2
Program python3 found: YES (/usr/bin/python)
Checking for function "atan2" with dependency -lm: YES 
Found CMake: /usr/bin/cmake (4.3.2)
Run-time dependency pthread found: NO  (tried pkg-config and cmake)
Run-time dependency pthreads found: NO  (tried pkg-config and cmake)
Run-time dependency threads found: YES
Has header "pthread.h" with dependency threads: YES 
Checking for function "pthread_create" with dependency threads: YES 
Has header "gmp.h" with dependency gmp: YES 
Has header "mpfr.h" with dependency mpfr: YES 
Checking if "GMP_LIMB_BITS is 64" with dependency gmp compiles: YES 
Checking if "mpz_init links" with dependency gmp links: YES 
Checking if "mpn_mul_basecase links" with dependency gmp links: YES 
Checking if "mpn_gcd_11 links" with dependency gmp links: YES 
Checking if "mpn_div_q links" with dependency gmp links: YES 
Checking if "mpn_add_n_sub_n links" with dependency gmp links: YES 
Checking if "mpn_add_nc links" with dependency gmp links: NO 
Checking if "mpn_addlsh1_n links" with dependency gmp links: YES 
Checking if "mpn_addlsh1_n_ip1 links" with dependency gmp links: NO 
Checking if "mpn_addmul_2 links" with dependency gmp links: YES 
Checking if "mpn_modexact_1_odd links" with dependency gmp links: NO 
Checking if "mpn_rsh1add_n links" with dependency gmp links: YES 
Checking if "mpn_rsh1sub_n links" with dependency gmp links: YES 
Checking if "mpn_sub_nc links" with dependency gmp links: NO 
Message: EXACT CPU: kabylake
Compiler for C supports arguments -Werror=implicit-function-declaration: YES 
Compiler for C supports arguments -Werror=newline-eof: NO 
Compiler for C supports arguments -Wno-stringop-overread: YES 
Compiler for C supports arguments -Wno-stringop-overflow: YES 
Compiler for C supports arguments -Wno-deprecated-declarations: YES 
Compiler for C supports arguments -march=skylake: YES 
Message: CPU-specific C arguments: -march=skylake
Message: FFT_SMALL:  enabled
Message: ASSEMBLY:  enabled
Message: ASM TEXT: ".text"
Message: ASM DATA: ".data"
Message: ASM LABEL_SUFFIX: ":"
Message: ASM GLOBL: ".globl"
Message: ASM GLOBL_ATTR: ""
Message: ASM GSYM_PREFIX: ""
Message: ASM RODATA: ".section  .rodata"
Message: ASM TYPE: ".type $1,@$2"
Message: ASM SIZE: ".size $1,$2"
Message: ASM LSYM_PREFIX: "L"
Message: ASM ALIGN_LOGARITHMIC: "no"
Message: ASM ALIGN_FILL_0x90: "yes"
Message: ASM HAVE_COFF_TYPE: "no"
Checking if "DIAGNOSTIC_PUSH pragma" compiles: YES 
Checking if "DIAGNOSTIC_POP pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_INCOMPATIBLE_FUNCTION_POINTER_TYPES pragma" compiles: NO 
Checking if "DIAGNOSTIC_IGNORE_DISCARDED_QUALIFIERS pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_FORMAT pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_DANGLING_POINTER pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_CAST_FUNCTION_TYPE pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_OVERLENGTH_STRINGS pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_UNUSED_VARIABLE pragma" compiles: YES 
Checking if "DIAGNOSTIC_IGNORE_MAYBE_UNINITIALIZED pragma" compiles: YES 
Checking if "PUSH_OPTIONS pragma" compiles: YES 
Checking if "POP_OPTIONS pragma" compiles: YES 
Checking if "OPTIMIZE_O2 pragma" compiles: YES 
Checking if "OPTIMIZE_OSIZE pragma" compiles: YES 
Checking if "OPTIMIZE_UNROLL_LOOPS pragma" compiles: YES 
Header "stdlib.h" has symbol "aligned_alloc" : YES 
Checking for function "_aligned_malloc" : NO 
Has header "stdarg.h" : YES 
Has header "math.h" : YES 
Has header "float.h" : YES 
Has header "errno.h" : YES 
Has header "alloca.h" : YES 
Has header "arm_neon.h" : NO 
Has header "dlfcn.h" : YES 
Has header "errno.h" : YES (cached)
Has header "fenv.h" : YES 
Has header "float.h" : YES (cached)
Has header "inttypes.h" : YES 
Has header "malloc.h" : YES 
Has header "math.h" : YES (cached)
Has header "pthread_np.h" : NO 
Has header "stdarg.h" : YES (cached)
Has header "stdint.h" : YES 
Has header "stdio.h" : YES 
Has header "stdlib.h" : YES 
Has header "string.h" : YES 
Has header "strings.h" : YES 
Has header "sys/param.h" : YES 
Has header "sys/stat.h" : YES 
Has header "sys/types.h" : YES 
Has header "unistd.h" : YES 
Has header "windows.h" : NO 
Configuring config.m4 using configuration
Program m4 found: YES (/usr/bin/m4)
Configuring flint.h using configuration
Configuring flint-config.h using configuration
Configuring config.h using configuration
Compiler for C supports arguments -march=native: YES 
Compiler for C supports arguments -Wall: YES 
Compiler for C supports arguments -Wextra: YES 
Compiler for C supports arguments -Werror: YES 
Found pkg-config: YES (/usr/bin/pkg-config) 2.5.1
Build targets in project: 315

Found ninja-1.13.2 at /usr/bin/ninja
$ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/lgoe/code/flint/build
ninja: Entering directory `/home/lgoe/code/flint/build'
[4172/5530] Compiling C object libflint.so.24.0.0.p/src_gr_generic_generic.c.o
In file included from ../src/gr_generic/generic.c:18:
../src/gr_generic/generic.c: In function ‘gr_generic_vec_dot_strided’:
include/flint/gr.h:1244:78: warning: ‘tmp1’ may be used uninitialized [-Wmaybe-uninitialized]
 1244 | #define GR_TMP_FREE(ptr, size) do { if ((size) > GR_TMP_VEC_ALLOC_MAX_STACK) flint_free(ptr); } while (0)
      |                                                                              ^~~~~~~~~~~~~~~
../src/gr_generic/generic.c:2544:13: note: in expansion of macro ‘GR_TMP_FREE’
 2544 |             GR_TMP_FREE(tmp1, len * sz);
      |             ^~~~~~~~~~~
../src/gr_generic/generic.c:2524:16: note: ‘tmp1’ was declared here
 2524 |         gr_ptr tmp1, tmp2;
      |                ^~~~
[4989/5530] Compiling C object libflint.so.24.0.0.p/src_nmod_poly_divrem_newton_n_preinv.c.o
../src/nmod_poly/divrem_newton_n_preinv.c: In function ‘_nmod_poly_divrem_try_sparse.isra’:
../src/nmod_poly/divrem_newton_n_preinv.c:54:9: warning: ‘coeffs’ may be used uninitialized [-Wmaybe-uninitialized]
   54 |         _nmod_vec_scalar_mul_nmod(coeffs, coeffs, nz - 1, Binv[0], mod);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../src/nmod_poly/divrem_newton_n_preinv.c:14:
include/flint/nmod_vec.h:120:6: note: by argument 2 of type ‘nn_srcptr’ {aka ‘const long unsigned int *’} to ‘_nmod_vec_scalar_mul_nmod’ declared here
  120 | void _nmod_vec_scalar_mul_nmod(nn_ptr res, nn_srcptr vec, slong len, ulong c, nmod_t mod);
      |      ^~~~~~~~~~~~~~~~~~~~~~~~~
../src/nmod_poly/divrem_newton_n_preinv.c:32:11: note: ‘coeffs’ declared here
   32 |     ulong coeffs[MAX_NZ];
      |           ^~~~~~
[5530/5530] Linking target libflint.so.24.0.0

@oscarbenjamin

Copy link
Copy Markdown
Contributor Author

As of the last commit pushed all CI jobs are using meson and are passing.

The means that the meson build is tested and seems to be working with various combinations of:

  • Linux, MacOS, Windows
  • x86, x86-64, arm64
  • gcc, clang, MSVC and MinGW.
  • Coverage measurement.
  • ASan build (this run showed a random ASan failure).
  • NTL, Nemo.jl, BLAS.

The examples can be built with (this is tested in CI):

meson setup build -Dexamples=enabled
meson test -C build --suite examples

I haven't looked at profiles and tune programs yet.

I'll have to take a closer look comparing the autotools build with the meson build to see if things like exact compiler flags are the same. I also should work through comparing where exactly the files go in the build directory for running the test executables and for linking ad-hoc programs in a development checkout.

@fredrik-johansson

Copy link
Copy Markdown
Collaborator

Noting that individual test program becomes build/src/acb/test/main instead of build/src/acb/test/main.

I don't see any difference between the two (@fredrik-johansson )

Sorry, I meant the current location is build/acb/test/main (no src).

@oscarbenjamin

Copy link
Copy Markdown
Contributor Author

Would it be worth it to add a light-weight CI job that just checks out the repository, runs python3 update_meson.py and fails if git status --porcelain has non-empty output?

Yes, definitely.

@oscarbenjamin

Copy link
Copy Markdown
Contributor Author
Run-time dependency pthread found: NO  (tried pkg-config and cmake)
Run-time dependency pthreads found: NO  (tried pkg-config and cmake)
Run-time dependency threads found: YES
Has header "pthread.h" with dependency threads: YES 
Checking for function "pthread_create" with dependency threads: YES 

I see now @lgoettgens how this works. The meson configuration here tries to use pkg-config but falls back on just trying to find the header. You can see this here:
https://github.com/oscarbenjamin/flint/blob/9958f448d1a0f0b91d415c2c5bc64e085d46b3dc/config/meson.build#L68-L91

  pthread_candidates += [
    dependency('pthread', required: false),
    dependency('pthreads', required: false),
    dependency('threads', required: false),
  ]

  foreach dep : pthread_candidates
    if dep.found() and cc.has_header('pthread.h', dependencies: dep) and cc.has_function('pthread_create', dependencies: dep)
      pthread_dep = dep
      have_pthread = true
      break
    endif
  endforeach

  if not have_pthread
    foreach libname : ['pthread', 'pthreadVC3', 'pthreadVCE3', 'pthreadVSE3']
      libpthread_dep = cc.find_library(libname, required: false)
      if libpthread_dep.found() and cc.has_header('pthread.h', dependencies: libpthread_dep) and cc.has_function('pthread_create', dependencies: libpthread_dep)
        pthread_dep = libpthread_dep
        have_pthread = true
        break
      endif
    endforeach
  endif

It looks like in your case the 'threads' dependency is found which I think is meson's generic version of having a thread library:
https://mesonbuild.com/Threads.html

Just using that might be better than trying to locate pthreads directly and having the fallback code that I made here but I'm not sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants