from Hacker News

Writing a Self-Mutating x86_64 C Program (2013)

by kepler471 on 5/25/25, 5:00 PM with 38 comments

• by ivanjermakov on 5/25/25, 8:09 PM

  I had a great experience writing self modified programs is a single instruction programming game SIC-1: https://store.steampowered.com/app/2124440/SIC1/

• by iamcreasy on 6/7/25, 2:28 AM

  Is it possible to mutate the text segment by another process? For example, injecting something malicious instead of exec-ing a shell?

• by Cloudef on 5/26/25, 11:17 AM

  Kaze Emanuar's "Optimizing with Bad Code" video also goes briefly go through self-modifying code https://www.youtube.com/watch?v=4LiP39gJuqE

• by DrZhvago on 5/26/25, 5:53 PM

  Someone correct me if I am wrong, but self-mutating code is not as uncommon as the author portrays it. I thought the whole idea of hotspot optimization in a compiler is essentially self-mutating code.

  Also, I spent a moderately successful internship at Microsoft working on dynamic assemblies. I never got deep enough into that to fully understand when and how customers where actually using it.

  https://learn.microsoft.com/en-us/dotnet/fundamentals/reflec...

• by pfdietz on 5/26/25, 11:56 AM

  A program that can generate, compile, and execute new code is nothing special in the Common Lisp world. One can build lambda expressions, invoke the compile function on them, and call the resulting compiled functions. One can even assign these functions to the symbol-function slot of symbols, allowing them to be called from pre-existing code that had been making calls to that function named by that symbol.

• by alcover on 5/25/25, 7:41 PM

  I often think this could maybe allow fantastic runtime optimisations. I realise this would be hardly debuggable but still..

• by xixixao on 5/26/25, 6:04 AM

  I’ve been thinking a lot about this topic lately, even studying how executables look on arm macOS. My motivation was exploring truly fast incremental compilation for native code.

  The only way to do this now on macOS is remapping whole pages as JIT. This makes it quite a challenge but still it might work…

• by oxcabe on 5/25/25, 8:06 PM

  It's impressive how well laid out the content in this article is. The spacing, tables, and code segments all look pristine to me, which is especially helpful given how dense and technical the content is.

• by belter on 5/25/25, 6:31 PM

  I guess in OpenBSD because of W ^ X this would not work?

• by Someone on 5/26/25, 6:01 AM

  Fun article, but the resulting code is extremely brittle:

  - assumes x86_64

  - makes the invalid assumption that functions get compiled into a contiguous range of bytes (I’m not aware of any compiler that violates that, but especially with profile-guided optimization or compilers that try to minimize program size, that may not be true, and there is nothing in the standard that guarantees it)

  - assumes (as the article acknowledges) that “to determine the length of foo(), we added an empty function, bar(), that immediately follows foo(). By subtracting the address of bar() from foo() we can determine the length in bytes of foo().”. Even simple “all functions align at cache lines” slightly violates that, and I can see a compiler or a linker move the otherwise unused bar away from foo for various reasons.

  - makes assumptions about the OS it is running on.

  - makes assumptions about the instructions that its source code gets compiled into. For example, in the original example, a sufficiently smart compiler could compile

    void foo(void) {
      int i=0;
      i++;
      printf("i: %d\n", i);
    }
  

  as

    void foo(void) {
      printf("1\n");
    }
  

  or maybe even

    void foo(void) {
      puts("1");
    }
  

  Changing compiler flags can already break this program.

  Also, why does this example work without flushing the instruction cache after modifying the code?