by gopiandcode on 6/17/25, 7:17 PM with 151 comments
by NathanKP on 6/17/25, 8:33 PM
To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's and frameworks. It isn't an unsolvable problem, particularly now that all the most popular coding agents have MCP support that allows you to bring in custom documentation context. However, there will always be a strong force in LLM's pushing users towards the runtimes and frameworks that have the most training data in the LLM.
by romaniv on 6/18/25, 2:35 PM
It is significant that LLMs in coding are being promoted based on a set of promises (and assumptions) that are getting instantly and completely reversed the moment the technology gets an iota of social adoption in some space.
"Everyone can code now!" -> "Everyone must learn a highly specialized set of techniques to prompt, test generated code, etc."
"LLMs are smart and can effortlessly interface with pre-existing technologies" -> "You must adopt these agent protocols, now"
"LLMs are great at 0-shot learning" -> "I will not use this language/library/version of tool, because my model isn't trained on its examples"
"LLMs effortlessly understand existing code" -> "You must change your code specifically to be understood by LLMs"
This is getting rather ridiculous.
by furyofantares on 6/17/25, 10:29 PM
> Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL.
I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. Agents even moreso if you can provide errors. I mean, I guess the article kinda goes in that direction.
But also authoring DSLs is something LLMs can assist with better than most programming tasks. LLMs are pretty great at producing code that's largely just a data pipeline.
by NiloCK on 6/18/25, 1:54 AM
At the time, I had given in to Claude 3.5's preference for python when spinning up my first substantive vibe-coded app. I'd never written a line of python before or since, but I just let the waves carry me. Claude and I vibed ourselves into a corner, and given my ignorance, I gave up on fixing things and declared the software done as-is. I'm now the proud owner of a tiny monstrosity that I completely depend on - my own local whisper dictation app with a system tray.
I've continued to think about stack ossification since. Still feels possible, given my recent frustration trying to use animejs v4 via an LLMs. There's a substantial api change between animejs v3 and v4, and no amount of direction or documentation placed in context could stop models from writing against the v3 api.
I see two ways out of the ossification attractor.
The obvious, passive, way out: frontier models cross a chasm with respect to 'putting aside' internalized knowledge (from the training data) in favor of in-context directions or some documentation-RAG solutions. I'm not terribly optimistic here - these models are hip-shooters by nature, and it feels to me that as they get smarter, this reflex feels stronger rather than weaker. Though: Sonnet 4 is generally a better instruction-follower than 3.7, so maybe.
The less obvious way out, which I hope someone is working on, is something like massive model-merging based on many cached micro fine-tunes against specific dependency versions, so that each workspace context can call out to modestly customized LLMs (LoRA style) where usage of incorrect versions of your dependencies has specifically been fine-tuned out.
by darepublic on 6/17/25, 8:07 PM
by jo32 on 6/18/25, 4:02 AM
by kibwen on 6/17/25, 9:49 PM
by usrbinbash on 6/18/25, 11:58 AM
DSL proliferation is a problem. I know this is not something many people care to hear, and I symphasize with that. Smart people are drawn to complexity and elegance, smart people like building solutions, and DSLs are complex and elegant solutions. I get it.
Problem is: Too many solutions create complexity, and complexity is the eternal enemy of [Grug][1]
Not every other problem domain needs its own language, and existing languages are designed to be adapted for many different problem domains. If LLMs help to stifle the wild growth of at least some DSLs that would otherwise be, then I am reasonably okay with that.
by adsharma on 6/18/25, 2:05 AM
This is what I've been focused on last few years with a bit of Direction 3 via
python -> smt2 -> z3 -> verified rust
requirements -> design -> design by contract -> subset of python -> gc capable language (a fork of golang with ML features?) -> low level compiled language (rust, zig or C++)
As you go from left to right, there is an increasing level of detail the programmer has to worry about. The trick is to pick the right level of detail for a task.
Previous writing: https://adsharma.github.io/agentic-transpilers/
by MoonGhost on 6/17/25, 8:47 PM
by maybevoid on 6/17/25, 9:59 PM
In an initial experiment, I found that LLMs could translate familiar shell scripting concepts into Hypershell syntax reasonably well. More interestingly, they were able to fix common issues like type mismatches, especially when given light guidance or examples. That’s a big deal, because, like many embedded DSLs, Hypershell produces verbose and noisy compiler errors. Surprisingly, the LLM could often identify the underlying cause hidden in that mess and make the right correction.
This opens up a compelling possibility: LLMs could help bridge the usability gap that often prevents embedded DSLs from being more widely adopted. Debuggability is often the Achilles' heel of such languages, and LLMs seem capable of mitigating that, at least in simple cases.
More broadly, I think DSLs are poised to play a much larger role in AI-assisted development. They can be designed to sit closer to natural language while retaining strong domain-specific semantics. And LLMs appear to pick them up quickly, as long as they're given the right examples or docs to work with.
by aaronvg on 6/17/25, 11:44 PM
Here's the cursor rules file we give folks: gist.github.com/aaronvg/b4f590f59b13dcfd79721239128ec208
by andybak on 6/18/25, 10:32 AM
LLMs just add another reason to this list.
by boznz on 6/17/25, 8:58 PM
* Not just English, substitute any other human language into the above
by api on 6/17/25, 8:51 PM
by amterp on 6/17/25, 10:03 PM
I don't blame anyone in the picture, I don't disagree that time saved with LLMs can be well worth it, but it still is a topic I think we in the PL community need to wrestle more with.
by kkukshtel on 6/17/25, 9:59 PM
What humans look at and what an AI looks at right now are similar only by circumstance, and what I sort of expect is that you start seeing something more like a "structure editor" that expresses underlying "dumb" code in a more abstract way such that humans can refactor it effectively, but what the human sees/edits isn't literally what the code "is".
IDK it's not written yet but when it is it will be here: https://kylekukshtel.com/llms-programming-language-design
by efitz on 6/18/25, 3:10 PM
Maybe DSLs are “write-only” languages for humans.
I don’t wish ill or sadness on anyone but it doesn’t bother me at all if LLMs drive DSLs into extinction.
by TeMPOraL on 6/17/25, 8:55 PM
So with LLMs making it easier to project back and forth between how programmer sees the task at hand, and the underlying dumb/straightforward code they ain't gonna read anyway, maybe we'll finally get to the point of addressing the actual problem of programming language design, which is that you cannot optimize for every task and cross-cutting concern at the same time and expect improvement across the board - we're already at the limit, we're just changing which time of day/part of the project will be more frustrating.
by jp57 on 6/17/25, 8:54 PM
I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to study programming languages for my PhD. Back then I was delighted by learning about cool new languages and language features.
But I did didn't study PL but rather ML, and then I went into industry and became a programming practitioner, rather than a PL researcher. I don't want a custom-designed ML programming language. I want a simple general-purpose language with good libraries that lets me quickly build the things I need to build. (coughPythoncoughcough)
Now that I have reached an age where I am aware of the finiteness of my time left in this universe, my reaction when I encounter cool new languages and language features now my is to wonder if they will be worth learning. Will the promised productivity gains allow me to recoup the cost of the time spent learning. My usual assessment is "probably not" (although now and then something worthwhile does come along).
I think that there is a very real chance that the idea of specialized programming languages will indeed disappear in the LLM era, as well as the need for various "ergonomic" features of general purpose languages that exist only to make it possible to express complex things in fewer lines of code. Will any of that be needed if the LLM can just write the code with what it has?
by cpard on 6/18/25, 4:07 AM
This is so true.
A couple months ago I was trying to use LLMs to come up with code to parse some semi-structured textual data based on a brief description from the user.
I didn't want to just ask the LLM to extract the information in a structured format as this would make it extremely slow when there's a lot data to parse.
My idea was, why not ask the LLM to come with a script that does the job. Kind of "compiling" what the user asks into a deterministic piece of code that will also be efficient. The LLM just has to figure out the structure and write some code to exploit it.
I also had the bright idea to define a DSL for parsing, instead of asking the LLM to write a python script. A simple DSL for a very specific task should be better than using something like Python in terms of generating correct scripts.
I defined the DSL, created the grammar and an interpreter and I started feeding the grammar definition to the LLM when I was prompting it to do the work I needed.
The result was underwhelming and also hilarious at some times. When I decided to build a loop and feed the model with the errors and ask to correct the script, I ended up sometimes having the model returning back python scripts, ignoring completely the instructions.
As the author said, everything is easier in Python, especially if you are a large language model!
by oleks on 6/17/25, 9:06 PM
by chr15m on 6/18/25, 1:05 AM
by noobermin on 6/18/25, 12:32 AM
If something is useful people will use it. Just because it seems like llms are everywhere, not everyone cares. I wouldn't want vibe coders to be my target audience anyway.
by kccqzy on 6/17/25, 9:48 PM
In the end I think mentioning Python is a red herring. You can produce an eDSL in Python that's not in LLM training data so difficult for LLMs to grok, and yet still perfectly valid Python. The deeper issue here is that even if you use Python, LLMs are restricting people to use a small subset of what Python is even capable of.
by ajross on 6/17/25, 8:24 PM
DSLs look great if they let you write the code you already know how to write faster. DSLs look like noise to everyone else, including Gemini and Claude.
I used to be a big DSL booster in my youth. No longer. Once you need to stop what you're doing and figure out your ninth or eleventh oddball syntax, you realize that (as per the article) Everything is Easier in Python.
by alganet on 6/18/25, 7:39 AM
Not necessarily true. There are two kinds of DSLs: external and internal.
An external DSL has its own tooling, parser, etc. The nix language, for example.
An internal DSL is like a small parasite that lives inside an existing language, reusing some of its syntax and tools. It's almost like intentional pareidolia. Like jQuery, for example.
Internal DSLs reduce the cognitive load, and in my opinion, they're the best kind of DSL.
by loa_in_ on 6/18/25, 1:48 PM
by keybored on 6/17/25, 9:44 PM
by waffletower on 6/18/25, 4:09 PM
by prats226 on 6/17/25, 9:18 PM
DSL's would be even harder for LLM's to get right in that case compared to the low-resource language itself
by outofpaper on 6/17/25, 10:45 PM
by Mathnerd314 on 6/17/25, 8:35 PM
But Python/Julia/Lua are by no means the most natural languages - what is natural is what people write before the LLM, the stuff that the LLM translates into Python. And it is hard to get a good look at these "raw prompts" as the LLM companies are keeping these datasets closely guarded, but from HumanEval and MBPP+ and YouTube videos of people vibe coding and such, it is clear that it is mostly English prose, with occasional formulas and code snippets thrown in, and also it is not "ugly" text but generally pre-processed through an LLM. So from my perspective the next step is to switch from Python as the source language to prompts as the source language - integrating LLM's into the compilation pipeline is a logical step. But, currently, they are too expensive to use consistently, so this is blocked by hardware development economics.
by wolpoli on 6/17/25, 11:51 PM
by jayd16 on 6/17/25, 8:11 PM
I suppose this could be done now for all the existing languages that target LLVM and unify the training set across languages.
by munificent on 6/17/25, 9:03 PM
As someone who loves a wide diversity of actively evolving programming languages, it makes me sad to think those days of innovation may be ending. But I hope that's not going to happen.
It has always been the case that anyone designing a new language or adding features to an existing one is acutely mindful of what programming language knowledge is already in the heads of their users. The reason so many languages, say, use `;` for statement terminators is not because that syntax is particularly beautiful. It's just familiar.
At the same time, designers assume that giving users a better way to express something may be worth the cost of asking them to learn and adapt to the new way.
In theory, that should be true of LLMs as well. Yes, a new language feature may be hard to get the LLM to auto-complete. But if human users find that feature makes their code easier to read and maintain, they still want to use it. They will, and eventually it will percolate out into the ecosystem to get picked up the next time the LLMs are trained, in the same way that human users learn new language features by stumbling onto it in code in the wild.
So I'd like to believe that we'll continue to be able to push languages forward even in a world where a large fraction of code is written by machines. I also hope that LLM training cost goes down and frequency goes up, so that the lag behind what's out there in the world and what the LLMs know gets smaller over time.
But it's definitely possible that instead of that, we'll get a feedback loop where human users don't know a language feature even exists because the LLMs never generate code using it, and the LLMs never learn the feature exists because humans aren't writing it.
I have this same fear about, well, basically everything with LLMs: an endless feedback loop where humans get their "information" from LLMs and churn out content which the LLMs train on and the whole world wanders off into a hallucinatory bubble no longer grounded in reality. I don't know how to get people and/or the LLMs to touch grass to avoid that.
I do hope I get to work on making languages great for humans first, and for LLMs second. I'm way more excited to go to work making something that actual living breathing people use than as input data for a giant soulless matrix of floats.
by jbellis on 6/17/25, 8:31 PM
by dack on 6/17/25, 8:37 PM
LLMs seem pretty good at figuring out these things when given a good feedback loop, and if the DSL truly makes complex programs easier to express, then LLMs could benefit from it too. Fewer lines of code can mean less context to write the program and understand it. But it has to be a good DSL and I wouldn't be surprised if many are just not worth it.
by nxobject on 6/18/25, 1:15 AM
by rramon on 6/17/25, 9:23 PM
by emiliobumachar on 6/18/25, 4:35 PM
by eru on 6/18/25, 1:32 AM
As AI systems improve, and especially as they add more 'self-play' in training, they might become really good at working in any language you can throw at some.
(To expand on the self-play aspect: when training you might want to create extra training data by randomly creating new 'fake' programming languages and letting it solve problems in them. It's just another way to add more training data.)
In any case, if you use an embedded DSL, like is already commonly done in Haskell, the LLMs should still give you good performance. In some sense, an 'embedded DSL' is just fancy name for a library in a specific style.
by msgodel on 6/18/25, 8:25 PM
by zzo38computer on 6/17/25, 8:48 PM
Also, domain-specific stuff can still be useful sometimes, and other stuff involved with designing a programming language.
by diimdeep on 6/17/25, 8:58 PM
by averkepasa on 6/17/25, 8:10 PM