from Hacker News

QUIC restarts, slow problems: udpgrm to the rescue

by jgrahamc on 5/7/25, 5:55 PM with 1 comments

• by majke on 5/8/25, 9:31 AM

  In this project I wrote quite some eBPF. I was constantly hitting verifier limits, and like everyone, I was initially just reordering variables and sprinkling "inline" or "noinline" everywhere. That wasn't sustainable.

  It turns out - there is now some reasonable tooling to understand verifier!

  For stack problems, clang accepts `-s` which prints stack requirement per function, like so:

    ** stack usage by function **
    ebpf/ebpf_aes128.c:180  AES_ECB_encrypt        32   static
    ebpf/ebpf_sha256.c:34   sha256_calc_chunk      64   static
    ebpf/ebpf_sha256.c:123  sha256_hmac            40   static
    ebpf/ebpf_quic.c:86     compute_hp_mask        24   static
    ebpf/ebpf_quic.c:242    decrypt_quic           16   static
    ebpf/ebpf_quic.c:193    _do_decrypt_quic_loop  16   static
  

  And for instruction count, I was able to feed the logs from verbose verifier (during loading) into code-coverage tooling, and count stuff up. The reuseorg prog takes 100k verifier instruction count/paths:

    ** verifier instruction count **
    udpgrm_reuseport_prog  processed  103486  insns  
    udpgrm_setsockopt      processed  9260    insns  
    udpgrm_getsockopt      processed  4215    insns  
    udpgrm_bpf_bind6       processed  75      insns
  

  While 100k is lower than 1m instr count limit, it's still a lot. And reordering some loop or introducing some "if" often makes that count baloon. Remember that verifier instructions is not real instructions during run. Rather it's a pessimistic interpretation of how many instr max under pessimistic conditions could possibly be run. I don't think having an actual run of that max is even practically possible. Think about it as upper bound of static analysis.

  Anyway - with stack and instr statistics it's way easier to make sense of verifier problems.