by phleet on 7/12/13, 3:24 PM with 169 comments
by VeejayRampay on 7/12/13, 4:56 PM
The "smartass" way of programming is sometimes overused but it does have its benefits. When you're metaprogramatically setting the attributes on the User, you're also avoiding needless and error-prone repetition and making sure that this central piece of code will either crash all the time or work all the time for all attributes. This has tremendous value.
So while I understand the point about this article, I might want to add a pinch of salt to the dogma underlying it.
by peterhunt on 7/12/13, 5:47 PM
Simply using it to save keystrokes is pretty lame since you'll spend much more time maintaining code than typing out the original and explicitness is valuable when returning to a piece of code. Additionally, if you find yourself that you need a lot of metaprogramming for a lot of things it's often an indication that you could just refactor your code using static idioms and be better off.
Not to mention that the more you use metaprogramming, the more likely it is that you or someone else will kill some runtime optimizations of the JIT.
IMO, syntax matters way less than people tend to think it does, and the additional implementation complexity and astonishment cute syntax introduces tends to make the trade-off not worth it.
by danso on 7/12/13, 4:35 PM
OK, sure, but after you've grokked the first "find_by" usecase...do you really need documentation for all the other kinds of "find_by"'s that you'll use?
In any case, I'd agree that meta-programming is too often abused, but the proposed grep test is far too strict. And, inability to create complete documentation for every single kind of method token is not really the main reason to avoid meta-programming...I'd say performance and the propensity for abuse are better motives.
This pull request re: removing most of Rails' dynamic finders in 4.x covers the topic nicely:
by pjungwir on 7/12/13, 4:24 PM
I also agree that Rails gets a pass. It's a little different when you have a stable (well, sort of :-) API with dozens of books and thousands of blog posts. It was annoying to learn what was going on at first, but that knowledge is more long-lasting so worth a bit more pain.
by stiff on 7/12/13, 7:34 PM
The general underlying problem is that a lot of people stick with those "principles" (which are rules of thumb, often weak ones at that) they have read somewhere without developing a real understanding of software design. Slogans like DRY or SRP seem to blind people to often simple forseeable consequences of their design decisions.
Software design boils down roughly to two abilities: one is imagining a great many ways of structuring a program, the other is the understanding of practical consequences of a given program structure. So a good software designer must ask him- or herself: why code repetition is a bad thing? And the answer is that there is absolutely _nothing_ wrong. What is wrong is that a logically or structurally common aspect of the problem didn't receive the recognition in the form of an abstraction (a method, class, variable, interface ...). Then you waste time by not being able to reuse this abstraction, when something about this common aspect changes you are forced to go through fifty places and modify the code, but the worst thing is that there is a limit of the amount of details a programmer can keep in his or her head, and the less you are able to structure your program the more restrictive will be limit of the problem complexity you will be able to tackle. That's the reason software design is at all important.
There are cases however when there is no real logical common denominator to two pieces of code and they are similar practically by accident. It is just as wrong to abstract an accidental similarity as it is not abstract an existing one; soon the requirements change and the abstraction will have to be abandoned, code copied and developed in divergent ways.
Finally, people start attempting metaprogramming way before they had learnt enough about structuring programs using basic means, like breaking things down into classes and methods appropriately, data-driving your programs etc. Yes, this is actually a skill, and unless you rewrite some of your own programs 5-10 times and compare their structure you won't learn it. I also recommend reading some classical books: SICP, Refactoring, Refactoring To Patterns, Effective Java, Programming Pearls.
And the ultimate lessons come from maintaining your own code for a few years.
by shurcooL on 7/12/13, 6:00 PM
Ok, it's actually right under one false assumption: that we are limited to using our existing text-based tools like text editors, grep and so on. As long as we use these existing tools, this advice is quite valid.
However, we are absolutely not limited to existing tools, we can and should make new ones that augment how we work and allow us to reach more dryness without compromises. You could have tools that let you work with ASTs, that expand code on the fly, show you higher level information, let you debug and step through sections of code effortlessly.
In the end, dryness is good, because it means you have less repetitive work to do when things change (and software is all about changing, unless you don't want to make progress). So it's very well worth investing into being able to achieve dryness more naturally without the downsides.
To quote Bret Victor, stop "blindly manipulating symbols." (You don't have to do do so overnight, just as long as it's a goal you work on achieving as time goes on.)
by forrestthewoods on 7/12/13, 4:59 PM
I work in C++ all day and loathe when functions are fully implemented in the class declaration. Keeping them seperate with full ClassName::FunctionName(...) scoping makes finding functions exceedingly simple and friendly. Keep implementations separate means the full class declaration is easy to read, parse, and understand. I don't want to scroll through hundreds of lines of code just to see what functions are available.
by pnathan on 7/12/13, 4:20 PM
A published & concise interface wins over an undocumented redundant interface. Explicit-only just takes you down the path towards Java and COBOL.
by calinet6 on 7/12/13, 5:09 PM
It's an artefact of the design. You simply need to know how the abstraction is done, and then you look for invocations of that abstract more general form. It's not that difficult, and if it's being done, it is almost without exception a better way to do the thing.
If metaprogramming is not a better way to do the thing being done, and adds confusion and decreases reusability, then you shouldn't do it, but that's a tautology and doesn't mean we have to be able to grep for everything we ever write.
by dfan on 7/12/13, 6:52 PM
by Periodic on 7/12/13, 4:52 PM
Other languages in which I've enjoyed REPLs include Python, JavaScript and Haskell. Two languages I have not found a good REPL for are Java and C++. Is anything of this sort available there?
by recursive on 7/12/13, 5:18 PM
by klochner on 7/12/13, 4:36 PM
- library code publishes an interface and can do what it wants
- your client/application code should be greppable
by agentultra on 7/12/13, 5:39 PM
The reason this difference is important is that metaprogramming is structured and explicitly defines the change in the program's semantics.
Thankfully I have never seen code like what was posted in my experience with Python so far. Explicit is better than implicit which the code example in this post would fail to pass IMO. Dynamically dispatching to names that don't exist in a class' published interface at call-time is a big no-no in my book.
Although practicality beats purity.
I haven't seen it yet but that doesn't mean there is a practical reason to use this method of dynamic dispatch. If there were one and it gets a problem solved NOW rather than waiting to find a better solution -- it might be worthwhile.
However it's a price you have to pay.
I think the grep-test is at least a good way to test the waters with a bit of code. I don't think it's a universal end-all-discussions rule.
by rralian on 7/12/13, 9:59 PM
by willurd on 7/12/13, 5:51 PM
This would also have the side benefit of being able to produce better designed search results.
EDIT: I suppose searching the AST wouldn't be enough, you would have to evaluate the code to some extent to be able to search these properties.
by zackbrown on 7/12/13, 7:05 PM
Imagine the 'incredible boon to productivity' you could get from tools (text editors, IDEs) that can deterministically show you all references or definitions of any method, variable, or class throughout your codebase whenever you want. (Hint: this isn't a dream--this is a huge plus of working with statically typed languages.)
For all of the pop-trendy love that dynamic languages seem to get for 'being fast for development,' (read: hacking) it's tragic how much they slow you down when you need to start hunting down where the hell something was magically (or meta-) declared or changed, especially when you're working with someone else's code (read: real life.)
The grep test seems like a great approach if you're stuck with a dynamic language. Of course, we don't always have the luxury to choose the technologies or platforms that we work with, but you've got the choice, a statically typed language solves this problem out of the box.
by mnarayan01 on 7/12/13, 4:55 PM
One thing I think the post might emphasize more thoroughly is that dynamic function invocation (at least when the function is defined in the project scope), is probably far more problematic than dynamic function declaration, particularly if sometimes the function is explicitly invoked, and sometimes dynamically. In this situation, I'll generally try to document that the function is dynamically invoked with the function declaration, but I'm always unsure what exact information I should put there. Simply saying something like "Dynamically invoked -- grep will not find all usages" is a minimum, but I often want to add more than just that.
by smrtinsert on 7/12/13, 4:14 PM
by toddkaufmann on 7/12/13, 7:44 PM
A better title would be "Don't use a fastening device if it fails the hammer test."
Are you don't use a tool that generates SetFunkyColumnName from funky_column_name data spec ? Or "I can't find the click handler in the .html, quit using jQuery" ?
I have been at the bottom of steep learning curves multiple times where I couldn't figure out where stuff was coming from. In most cases I got over it
Use the appropriate tools that get the job done and make you and your team productive. Tools and ideas evolve at different rates. Not all tools or ideas are implemented properly the first time, and not all tools or ideas are necessarily good/useful. But we'll never get any further if we don't try...
by dbloom on 7/12/13, 5:18 PM
So in this case, writing longer, "grep-friendly" JS code can actually reduce the size of the JS payload you serve to your users.
by wizzard on 7/12/13, 5:42 PM
It's madness to classify all use of metaprogramming as abuse. Perhaps you need to pour a glass of wine and learn to savor the source code, and to appreciate the power of modern programming languages.
by swah on 7/12/13, 5:29 PM
(See Yegge's grok project, unfortunately he doesn't blog anymore)
by TylerE on 7/12/13, 4:13 PM
by gwu78 on 7/12/13, 8:50 PM
The idea is that programming languages allow too much flexibility in form and syntax for any indexing system to accomodate the full range of possibilities. And "coding style" rules are apparently too restrictive on creativity: they are too difficult to enforce. And hence a solution would be better to focus not on the code, but on the comments. Force programmers to adhere to a uniform commenting system.
The OP mentions ctags. It's not a perfect system, but it's still in the BSD base systems so everyone who has BSD has a copy of the needed programs. That's a start.
What about cxref? Another old system that's probably not perfect, but seems like it was aiming in the right direction.
I've never understood why programmers obsess about things like verbose function names (that make lines go way over 80 chars and bend across the page... with identation it becomes almost unreadable to my eyes) instead of just providing an index of all functions and including the verbose information in the index, not the code. vi (and no doubt emacs too) allows you to jump around easily so you could look things up in an index quite quickly.
Why do Wikipedia pages have an index of footnotes and references at the bottom? Why not stuff all this information into the words in the body of the article? Why do books have indexes? Why do academic papers use footnotes? I don't know. But I'm accustomed to these conventions.
I also don't know why code uses verbose function names and generally lacks an index or footnotes. But I guess programmers have just become accustomed to these conventions.
by ascotan on 7/12/13, 9:19 PM
This entire post is silly.
1. Not being able to grep for code has nothing to do with being DRY.
2. Being able to grep for something doesn't make it correct. Nor does it make it less maintainable if you don't used named functions.
Moving on...
by jimmaswell on 7/12/13, 5:59 PM
by michaelfeathers on 7/12/13, 8:44 PM
The idea is that when you look at code, the computations you see are the ones actually executed. Meta-programming may be used but only to add behavior to existing code, not to nullify it.
A good example is an 'execute around' method. The method you see in the code is executed, but some things may happen before it and some things may happen after it. What you can't do is replace the body with something else.
An interesting thing about the 'No Lie Principle' is that aligns with good practice around inheritance also. It's better to override abstract methods than it is to override concrete ones for a number of reasons.
by Arnor on 7/12/13, 7:08 PM
function do($class, $action, $param, $value) {
$method = $action.$param
$obj = new $class();
$obj->$method($value);
return $obj
}
$my_name = 'Arnor';
$$my_name = do('Entity', 'set', 'Name', $my_name);
// The entity is now stored in the variable $Arnor
It's fun to come up with clever solutions and puzzles, but it harms your code base. If you're proud of how new and clever your last 10 lines were, you probably need to refactor it.
by themstheones on 7/12/13, 4:41 PM
by gohrt on 7/12/13, 9:43 PM
For example, Guice fails the Grep Test hard, but it incredibly helpful.
All data-driven fail the Grep test. Your browser fails the Grep test (you can't grep for javascript content)
You just need to replace Grep with an xref tool that understands your programming language, including your metaprogramming. This may require you to commit your configuration files and standard data objects into your source control, or build an indexer that can read your CMS as well as your code.
by ilcavero on 7/12/13, 4:18 PM
by flashmob on 7/12/13, 5:42 PM
by jol on 7/12/13, 4:09 PM
by rocky1138 on 7/12/13, 5:03 PM
by ColinWright on 7/12/13, 5:19 PM
It's a difficult trade-off, one one I've had to make calls on several times. Do you use the full capabilities of incredibly gifted and talented programmers, and then allow code into your codebase that maintenance programmers can't understand?
Tricky.
by dllthomas on 7/13/13, 2:31 PM
by wpeterson on 7/12/13, 10:50 PM
There's nothing wrong with using metaprogramming to generate methods, as long as you are writing tests for those methods.
Grepping the application code is usually much less useful than grepping the tests to see how the system is intended to behave.
by softbuilder on 7/12/13, 7:29 PM
by aaronblohowiak on 7/12/13, 6:22 PM
by chameco on 7/12/13, 5:48 PM
by serichsen on 7/12/13, 8:10 PM
by halayli on 7/12/13, 11:37 PM
by jamii on 7/12/13, 8:25 PM
user=> (meta #'leiningen.gnome/uuid)
{:arglists ([project]), :ns #<Namespace leiningen.gnome>, :name uuid, :column 1, :line 10, :file "leiningen/gnome.clj"}by penguindev on 7/12/13, 4:34 PM
by jfarmer on 7/13/13, 1:54 AM
The point of DRY isn't to mindlessly remove code duplication. It's to remind us to look for code duplication and keep us mindful of coupling between the various parts of our code.
Where two identical chunks of code that represent the same kind of work are used in multiple places we introduce "algorithmic coupling." That is, whenever the work being done in one location changes, we have to make sure to change the work being done in the other location. Anyone reading this code -- whether the author, the author's future self, or teammates -- has to remember this extra fact, increasing the surface area for bugs.
There's also "name coupling," viz., for every vector _foo_ associated with the Ray we want methods like foo_is_zero?, negative_foo, etc. Here it's important that the naming convention be consistent, so there's coupling there, too. If the names aren't consistent anyone else reading the code would then have to remember that fact, and anyone changing the names would have to remember to update all the other names, too.
The irony of his example is that style of metaprogramming is a great way to get rid of name coupling, but he did nothing to get rid of the much worse algorithmic coupling. Indeed, algorithmic coupling screams for DRY whereas name coupling requires it on a more case-by-case basis.
That is, when all the names are highly localized, e.g., three short methods that share some naming pattern are all defined in succession, it's much less important to remove the duplication. Anyone reading or editing that code will quickly see what the pattern is and why it exists.
Here's a comment I left on the blog:
Hmm. I don't think the lesson here is about to-DRY-or-not-to-DRY your code. Instead, it's about using the appropriate abstractions.
Using the Ray example, both position and direction are vectors, not points. Make them vectors! Then you'd be able to say things like
ray.position.zero?
# We'd usually say "the inverse of" not "the negative of"
ray.position.inverse
class Ray
def position_is_zero?
position.zero?
end
def direction_is_zero?
direction.zero?
end
end
In a world where all of your logic is also duplicated in the Ray class, the code would break when either the names or the logic changed.
by fleitz on 7/12/13, 5:32 PM
Making code that passes the grep test allows many more programmers who are vaguely familiar with the codebase to make changes.
Making code that fails the grep test allows a team of a few highly skilled developers who know the codebase inside and out to do the work of hundreds.
It's like mathematical notation, you generally need experience in that sub-branch of mathematics to understand the notation.
by leifaffles on 7/12/13, 7:15 PM
Much like languages have more successful ways of representing numbers, some languages are more successful at metaprogramming. In particular, Lisp takes the view that the act of metaprogramming is really the act of writing application-specific compiler extensions. The default behavior simply doesn't involve ripping apart strings and concating them back together.
by dschiptsov on 7/13/13, 8:01 AM
The practice of placing any general functionality in lambdas or blocks will make thing more messy.
by AsymetricCom on 7/12/13, 10:43 PM