from Hacker News

Pair Your Compilers at the ABI Café

by nrabulinski on 5/6/24, 6:35 AM with 37 comments

  • by jcranmer on 5/8/24, 4:52 PM

    I can definitely feel the pain of trying to work out ABI mismatch concerns. It doesn't help that it often isn't clear from the output what some of the underlying assumptions are--expected stack alignment, for example, or structs being broken up into registers or not.

    It would be nice if compilers could output some sort of metadata that basically says "ah yes, here's a struct, it requires this alignment, fields are at these offsets and have these sizes, and are present in these cases" (the latter option being able to support discriminated unions) or "function call, parameters are here, here, and here." You'd think this is what DWARF itself provides, but if you play around with DWARF for a bit, you discover that it actually lacks a lot of the low-level ABI details you want to uncover; instead, it's more of a format that's meant to be generic enough to convey to the debugger the AST of the source program along with some hint of how to map the binary code to that AST--you can't really write a language-agnostic DWARF-based debugger.

  • by zX41ZdbW on 5/8/24, 7:54 PM

    I think the right way to avoid this problem is to avoid using ABI at runtime or build time.

    At runtime, it means - don't use shared libraries. At build time, it means - build every library from the source, don't use pre-built artifacts.

    This sounds controversial... But it allows you to change compiler or compiler options at any time, and you don't have to bother. It also enables cross-compilation, reproducible builds, and portable binaries. You no longer have to ask developers to set up a complex build environment on a specific Linux distribution because it works everywhere.

    I use this approach for ClickHouse.

  • by not2b on 5/8/24, 4:21 PM

    There is a specified common C++ ABI that gcc, clang, Intel's proprietary compiler, and others use. It was originally developed for the Itanium processor but is now used by gcc and clang for everything. See

    https://itanium-cxx-abi.github.io/cxx-abi/abi.html

    Unfortunately this ABI didn't specify how __int128 (and other nonstandard types) are to be passed.

  • by gigel82 on 5/8/24, 9:00 PM

    I struggled with this many times and at the end of the day threw down the towel and just wrapped everything in plain C exports. That's the only way I know to get ABI compatibility across different compilers/toolsets/versions. COM-like constructs come as a close second.

    It's an unfortunate state.

  • by w10-1 on 5/8/24, 5:19 PM

    Also function pointers, errors & exception-handling, async/channels/thread-local's, go stacks, swift @objc, @cdecl and cpp inter-op, FFI dialects...

    It's not really pain anymore; it's a kind of hilarity

  • by tialaramex on 5/9/24, 9:42 AM

    If I understand correctly there's also an ABI problem for synchronization rules.

    Within a compiler, it actually doesn't matter whether we think about a problem as A mustn't happen after B, or as B mustn't happen before A. But expressing this across an ABI we have to be careful that we don't have buck passing. Suppose Language #1 thinks of it the first way, and Language #2 the second way, now if Language #1 is responsible for B while Language #2 is responsible for A, each may believe the other will have taken care of ordering and no synchronization is actually implemented.

    Overall, "ABI" turns out to mean something like "Every assumption you've made which can be detected by other software, including assumptions you didn't realise you had". Discovering all your assumptions is hard, accepting that other people assumed different and they aren't just wrong is also surprisingly hard.

  • by gumby on 5/8/24, 3:22 PM

    It's Interop for compilers!

  • by fowl2 on 5/13/24, 10:56 AM

    I mean there's tons of IDLs, so many no one can agree on which one!

    oblig: https://xkcd.com/927/