Producing FFI static binaries

[andfoy]

Hi, thanks for your work on this!

I have a question regarding to the production of FFI binaries on Haskell using nix. Right now I'm trying to compile this project, https://github.com/treble-ai/duckling-ffi, which intends to produce a C-compliant static library. Until now, Nix is able to build the library and produces a static binary libHSduckling-ffi-0.1.0.0-28aDIsnrQtXFlJL0Bi0HJ4.a. However, when I inspect that binary using nm, I find that several Haskell runtime symbols are undefined, such as base_GHCziForeign_zdwpeekCString_closure, or rts_getPtr, which prevent the output library to be useful. Is there any way to produce a full, static binary using static-haskell-nix? Sorry for my lack of background, as I don't have any previous haskell/nix experience.

[nh2]

static binary libHSduckling-ffi-0.1.0.0-28aDIsnrQtXFlJL0Bi0HJ4.a.

There may be a confusion in there, usually people refer to .a files as "static library" not "static binary"; usually "static binary" refers to an (e.g. ELF) executable file that has no dynamic loading dependencies (as shown by ldd).

Separately:

I suspect that libHSduckling will contain only code that's from your library. You have to link with the .a files of your dependencies (e.g. .base and the RTS) to get all symbols satisfied. That is usually what GHC does when you build the final executable (linking together all the .a files into one exe).

[andfoy]

@nh2, thanks for the quick response, do I need to update anything on the default.nix file? I tried to add the staticlib flag to ghc-options, but I don't see that the output library being produced anywhere

[nh2]

I think we need to backtrack a bit first so that I can understand what you want to achieve.

What I said above boils down to "it is normal that your libHSduckling-ffi-0.1.0.0-28aDIsnrQtXFlJL0Bi0HJ4.a contains undefined symbols; they are resolved at executable linking time".

Now, what do you want to do? You're saying to

produce a C-compliant static library

From looking at https://github.com/facebook/duckling, it seems that it's a Haskell-only library, so I suspect you "C-compliant" means that you want to be able to do gcc yourprogram.c -l:ducklingWithAllDependencies.a (or similarly, link it into another compiled language like C++ or Go), where ducklingWithAllDependencies.a has the duckling library itself as well libHSbase....a and the GHC runtime as well?

If yes, then I have a good and a bad message:

• Bad: I don't actually know much about combined .a "mega-archives", because I so far have dealt mainly with static executables, for which it's fine to have all the libs as separate .as and then link them together.

• Good: Others seem to have implemented high-level tooling support for that, in particular @angerman in Add staticlib haskell/cabal#4617. That adds https://www.haskell.org/cabal/release/latest/doc/users-guide/nix-local-build.html#static-linking-options:

  Roll this and all dependent libraries into a combined .a archive. This uses GHCs -staticlib flag,

  and apparently you can set static: true in your cabal file (pass --enable-static to cabal configure when running it manually).
  You may not even need nix / static-haskell-nix for that.

(I notice you use ghc-options: -static-library in your cabal file; perhaps using cabal's higher-level static: true gets you further -- let me know!)

[andfoy]

From looking at https://github.com/facebook/duckling, it seems that it's a Haskell-only library, so I suspect you "C-compliant" means that you want to be able to do gcc yourprogram.c -l:ducklingWithAllDependencies.a (or similarly, link it into another compiled language like C++ or Go), where ducklingWithAllDependencies.a has the duckling library itself as well libHSbase....a and the GHC runtime as well?

That's resumes basically what I'm trying to do! I'll try to use static: true, does that need to have a statically-linked Haskell?

[nh2]

does that need to have a statically-linked Haskell?

I'd guess not, but I discovered static: true also only today, so I cannot be sure.

[angerman]

Just a few notes, as this showed up due to the mention:

• You do not necessarily need aggregated archives as libraries. They are after all just archives of object files. You can also just ship the haskell .a and the dependent .a. That is just ship multiple archives. That's what you do when you link the library in the end with -l<name> anyway.
• Archives have the annoying ability to store multiple objects with the same name.
• -staticlib uses the Ar module in GHC to just concatenate the archives ghc knows about. You can do this with MRI scripts and gnu ar as well, but as GHC tries to not rely too much on the tools available and gnu ar, bsd ar, and version are just a mess it ships it's own Ar module. The format is trivial enough that having our own module is easier than dealing with the mess that the tools are.

Do you need a statically linked GHC? No. Do you need the dependent archives available to roll them into a combined archive. Well, yes. Does GHC usually ship with the archives? Yes.

[andfoy]

@angerman, thanks for the detailed info. I have a question regarding fPIC, I have been not able to compile my library, as the linker complains that many of the core Haskell library were not compiled using fPIC. Which is the correct way of doing it?

[andfoy]

I managed to produce a static library, however, I get undefined symbols such as base_GHCziBase_eqString_info. I tried to add HSbase to the extra-libraries flag in the cabal config file, but the result is the same

[angerman]

How exactly did you build it? Did it end up calling GHc with -staticlib?

[andfoy]

@angerman, yes indeed, I added -staticlib

[angerman]

You might have to look at the verbose output (-v3 to figure out what’s going on then :-/

You could always link against libHSbase...a as well I think. But staticlib should have rolled that into the final output.