from Hacker News

The Marshmallow Test does not reliably predict adult functioning

by superposeur on 8/2/24, 3:49 PM with 400 comments

by rahimnathwani on 8/2/24, 5:39 PM
The findings of the original study were called into question by a larger 2018 study[0]. The original study had 90 students. Some folks did a study with 900 people. They found the same correlation that the original study did. But when they controlled for household income, they found most of the correlation disappeared.
The obvious conclusion is that household income is a predictor of both:
- inability to delay gratification, and
- higher academic achievement
This makes sense when you consider that someone growing up in a poor household may have both:
- less reliable/continuous/predictable access to material things, meaning they would rationally seize immediate opportunities rather than taking the risk of a larger future opportunity, and
- less academic support
Now, this new study (OP) goes even further, finding that the correlation itself is weak.
[0] Watts, T. W., Duncan, G. J., & Quan, H. (2018). Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Early Delay of Gratification and Later Outcomes. Psychological Science, 29(7), 1159-1177. https://doi.org/10.1177/0956797618761661
by dekhn on 8/2/24, 5:44 PM
Many details of this particular experiment made me greatly reduce my confidence and interest in social science. I was trained up in quantitative biology- and when I look at studies like this, I see a long list of "things that could go wrong, leading the investigator to falsely conclude their hypothesis is true". But in this case, I think the investigator actually didn't care enough about doing high quality research- they simply started with a moral belief/value judgement and ran an experiment and chose to interpret the results to support their "hypothesis". And the nature of social science is such that it's really hard to truly run an "honest experiment".
by jmugan on 8/2/24, 5:34 PM
It's funny. When you first do work, you want the experiment to satisfy your hypothesis. When you are building on work, you also want the replication to succeed. But when it is a famous result like this, you actually want it to fail so people talk about your result. There are uncountable ways that these experiments can be unconsciously and subtly affected by the desire of the experimenter.
As an aside, I believe one interesting confounder in the marshmallow test is that it tests more (or at least as much) the subject's trust that the eventual reward will actually be given as it does the subject's ability to wait for the reward. So if you live in an unpredictable environment, it's better to just eat it.
by sunjieming on 8/2/24, 7:00 PM
Virtually every study I read about in AP Psych in HS failed to replicate - including this one. That whole class in hindsight was at best a waste of time and at worst provided bad info to make life decisions on
by luketheobscure on 8/2/24, 5:36 PM
An alternative interpretation of the Marshmallow Test is that it is a measurement of trust as much as it is of self control. If you don't believe that the researchers are going to give you the two marshmallows, then you're not going to wait.
by niemandhier on 8/2/24, 9:10 PM
Children that trust the adult making the promise tend to be able to delay their own gratification:
https://pubmed.ncbi.nlm.nih.gov/26799458/
From a game theoretic point of view it makes sense:
If your internal model of adults suggests, that you should put a gausian prior on the waiting time until they keep their promise, i.e. most adults in you life tend to keep their word, waiting makes sense.
If however your experience tells you to assume a power law as prior, cutting you losses after a time is perfectly rational.
This has a certain beauty, since it would mean that success in life correlates with dependable parents and given the temporal component I actually would assume causality.
by influx on 8/2/24, 5:17 PM
Have any famous psychological tests replicated?
by jti107 on 8/2/24, 5:20 PM
anedotally this has held up in my social group. the people that i grew up with and went to school with...the ones that could delay instant gratification and had long term goals ended up doing pretty well in life. the ones that didnt have any plans and just went with the flow did poorly and just getting by.
also in my life i notice a big difference in performance from when i had goals/vision for my life vs. going through the motions.
IMO i think you need to have goals/vision/standards for all the important areas in your life (hobbies,partner,career,family,relationships)
by FredPret on 8/2/24, 6:23 PM
Some fields of study will always be art, not science.
Literature, art, human psychology. A good writer, artist, or therapist can make a truly great contribution. But they cannot conduct disciplined experiments and establish truth numerically.
And that is OK.
What is not OK is the cabal of academic psychologists who don’t even know that they’re full of shit because they aren’t trained in any of the numerical / “hard” disciplines. (Hard as in well-defined, not difficult).
by Mozai on 8/2/24, 8:33 PM
I remember growing up getting into scenarios like the Marshmellow Test, but I didn't learn to delay gratification; what I learned was I'm a sucker if I wait or make sacrifices. Often "you'll get two later if you surrender this one now" became "there is no second marshmellow and you're not getting the first one back." How many times do other kids have to experience this before they learn not to delay gratification? and thus get accused of "poor impulse control" when I'd call it "learning from experience" ?
by suzzer99 on 8/2/24, 5:27 PM
I've always suspected the marshmallow test measures desire to please the researcher more than anything else.
I'm supposed to sit here and stare at this marshmallow for some indeterminate amount of time, just to get one more marshmallow? Offer me a whole bag and we'll talk. Otherwise, you're wasting my time. My marshmallow would be gone before they could finish explaining the task.
by spiderice on 8/2/24, 7:26 PM
When Dieter Uchtdorf was in the presidency of the LDS branch of Mormonism he gave a talk to the entire church about this study. It’s since basically become doctrine in the LDS church. Funny how far and widespread these inaccurate studies can become. And the large majority of the people who hear the original study will never hear that it wasn’t reliable.
by superposeur on 8/2/24, 3:52 PM
Thank god, as I love marshmallows and instant gratification.
by helsinkiandrew on 8/2/24, 5:25 PM
Found this article whilst looking for more details, the same results seem to have been reported for several years, including following the subjects into middle age:
https://anderson-review.ucla.edu/new-study-disavows-marshmal...
> As the researchers predicted, the study finds only a tiny correlation between marshmallow test times and midlife capital formation. A graduate’s score on the self-regulation index was, however, modestly predictive of their middle-age capital formation, the study finds.
by brnaftr361 on 8/3/24, 4:59 PM
I expect a contributing factor to this is natural inference. Delayed gratification is just fine for certain windows and in certain domains. But continuous delay (or non-gratifying outcomes) are surefire ends to update priors. Anecdotally, I'm very much in that position. I've delayed and subsequently mistimed my whole life and let's just say my expectations are totally unmet and I'm quickly unraveling into a hedonist plus flagellant.
I'd expect a decay of delayed gratification in aggregate. And this will vary from individual to individual dependant on their expectations/(negative realizations - positive realizations) or similar, and negative realizations are supposedly weighted higher than positive by a factor of 3-5. This exacerbates the rapidity of decay.
I'd posit, then, that delayed gratification can predict within a window; that window may be a "critical window" which leads to enhanced success. Failing to obtain that success then predicts regular decrements to delayed gratification metrics.
And delayed gratification isn't beneficial in all scenarios anyways. Sometimes the payoff is in immediate and remorseless action.
by charlie0 on 8/2/24, 5:32 PM
Maybe tempting children with marshmallows is a bad proxy for testing delayed gratification, but the thesis about being able to delay gratification leading to success seems to be true as far as I can tell. Anecdotally, all the people I know who can't delay gratification are just scraping by (this includes another SWE who earns a decent amount but is rather impulsive). All those I know who can delay it are doing great.
by m3kw9 on 8/2/24, 5:13 PM
Maybe when they grow up some of them learned to steer away from instant gratification. Or maybe you need to account for how big luck is in the success in life
by jpwagner on 8/2/24, 5:20 PM
Think of the marshmallow test as a short story by a famous author. It rings with truth, but it's not "science".
by tibbon on 8/3/24, 11:38 AM
It is interesting to read something like this and then go back to YouTube where there are 100s of videos pointing to this test as one of our most important ways to understand psychology and success. I suppose we all parrot things that have little basis in reality and we have not verified for ourselves
by rerdavies on 8/3/24, 10:13 AM
Value of one extra marshmallow: 4 cents.
Time it takes to earn an extra marshmallow: 20 minutes.
Hourly earned value (assuming you like marshmallows): 12 cents.
Reasons not to like marshmallows: The principal ingredient is gelatin, a protein obtained by boiling skin, tendons, ligaments, and/or bones with water. And they don't really taste that great.
It has always seemed to me that the best strategy in this situation is to eat the marshmallow right away in the hopes that the psychologists will let you out of the room early. A better strategy might be to refuse to stay in the room for 20 minutes.
by jgalt212 on 8/2/24, 7:43 PM
I think all of these studies fail to account for the credit component. i.e. I can see that this man I just met has one cookie, but now he's promising me another cookie (which I may or may not be able to see). And then if I do what he says, he'll give me two cookies. What probability do I assign to the chances he can deliver on his promises? Maybe he's a liar. Maybe before I completed the assigned task, he came across a better deal and allocated all his cookies.
by andrewp123 on 8/3/24, 6:23 PM
I don’t know how any experiment like this could be taken seriously. Your action could change 15 minutes apart if you feel like having sugar, if the guy felt intimidating or not, if the last thing he said seemed friendly or if his facial expression was off at the very end, etc.
There must be a better way of judging the validity of a social experiment using first principles. There’s a huge psychological side that people completely ignore.
by PaulHoule on 8/2/24, 5:55 PM
I'd like to see the 2024 version where the kid who got two marshmallows is fat and the one who didn't want any marshmallows at all is skinny.
by marcell on 8/2/24, 11:00 PM
The study found associations at r=0.17 for both BMI and educational attainment. Not a lot but not zero.
The marshmallow test deals with kids so it’s noisy by nature, that there are two mild associations is interesting. It has mild predictive value.
I think there’s a strong desire to have this test shown to be faulty. Perhaps because the test is so easy to do, parents do it on their own kids and don’t like the outcome.
by jwie on 8/2/24, 11:03 PM
Marshmallow tests are more a test of the child’s priors about adult reliability.
If the child has reliable parents they tend to pass the test. The children of reliable parents do better in life, which is obvious.
The test also fails to account for a temperate child that doesn’t actually want more than one in the first place and isn’t playing the researchers game.
by tqi on 8/2/24, 7:27 PM
I've never understood the "so what" of this study. Did people not think self control was a virtuous characteristic before? Will they stop trying to teach their kids to exercise self control now that it's been debunked?
Sometimes it feels like much of social psychology exists primarily to sell books and lecture series tickets.
by siilats on 8/3/24, 4:49 PM
So there are two options. You get a coefficient of 0.2 and a std error of 0.2 so you say it’s not significant but the reason is you don’t have enough data so st error is too large. Or you have a coefficient of 0.0001 and a st error of 0.01 so you are pretty sure there is no relationship.
by wanderingmind on 8/3/24, 4:19 AM
Hidden Brain had an episode on the recent studies challenging the Marshmallow test https://hiddenbrain.org/podcast/when-to-eat-the-marshmallow/
by bell-cot on 8/2/24, 5:59 PM
Imagined childish reasoning: I could eat one marshmallow now, and hopefully finish this stupidboringweird test sooner and go home. Or I could be stuck here longer, for one crappy little marshmallow, showing that I know how to play a stupid suck-up little teacher's pet.
by paganel on 8/2/24, 5:55 PM
By this point all the normal people have started ignoring this type of “science”, many of us were ignoring it from the very beginning. Too bad that this quackery has already made its way into many States’ apparatuses, see the obsession about the nudge thing, for example.
by honkycat on 8/2/24, 7:33 PM
I've always thought this was stupid and obviously not real.
What if the child was being playful by not following the obvious "correct" path? Wouldn't that point to someone who is social and humorous and happy? Isn't that an advantage?
by rolph on 8/2/24, 5:24 PM
when marshmallow tested, i spit on my marshmallow, when asked why i explained "now noone will want that one", "now i get two marshmallows because i waited" , "and also a third one cause only i will want it"
by PopePompus on 8/3/24, 2:13 AM
I think eventually the only famous Psych. experiment which will remain undebunked is the Milgram Experiment, not because it is better than the rest, but because nobody would be allowed to try to reproduce it today.
by bradgessler on 8/3/24, 4:34 AM
The results of this study have always bothered me.
I don’t like marshmallows. Never have. If I was run through the marshmallow test I would have done whatever it takes to get out of there quickly and not have to eat marshmallows.
by aristofun on 8/2/24, 9:41 PM
No wonder, generally speaking. Human nature is way deeper and more complex, more fluid than any artificial model/framework imposed on it.
Psychology is not a reproducible science strictly speaking for that reason.
by WhitneyLand on 8/2/24, 5:54 PM
So in a nutshell, one of the greatest failings of science in history comes down to, researchers were under pressure so they caved and compromise their ethics and morals.
Even worse, the replication crisis is only one reason that the public has continued to lose faith in science in the post truth era.
It’s also the disinformation campaigns that set out to attack whatever’s in a groups interest whether it be politics or the environment.
Maybe the coup de grâce will be social media which encapsulates people into bubbles seemingly impenetrable to the truth.
by yodon on 8/2/24, 5:43 PM
I can speak to this test a bit from experience: as a very young child, I was in a pilot study used to design a large longitudinal study, and my younger sibling was in that large longitudinal study.
At about age 4, I ended up literally maxing out the delayed gratification test and being sent home with a ridiculously large bag of M&M's, much to this dismay of my mom.
With that as context, I wonder whether some of the changes/lack of reproducibility are actually measures of decreasing economic mobility and economic agency within the US.
Early studies on ability to delay gratification were done during the favorable economic conditions baby boomers grew up in. More recent studies were done in eras with far less economic mobility.
It's quite likely you'd see a smaller effect today, not because the impact isn't there, but because it's so much harder today to make a significant upward change in your economic status.
by xbar on 8/2/24, 10:28 PM
Calling it a test is almost certainly an exaggeration at this point.
Perhaps we could call it "The Marshmallow Trick" now?
by parkaboy on 8/3/24, 1:51 AM
I was hoping (based on my initial reading of the HN title) that they tried the Marshmallow test on adults.
by Banditoz on 8/2/24, 6:22 PM
I'm confused. How do you access the full text of the article? Why is it behind a $15 charge?
by idunnoman1222 on 8/3/24, 3:06 AM
Once we control for X
You can control away anything the whole idea of isolation is bunk
by sandspar on 8/2/24, 8:08 PM
>dynamite psychology result with far reaching conclusions fails to replicate
No way?
by lawlessone on 8/2/24, 6:57 PM
Try it on shareholders.
by FrustratedMonky on 8/2/24, 7:37 PM
Wut? I gave up all those marshmallows for nothing?
by aqsalose on 8/2/24, 5:59 PM
From abstract (article is paywalled)
>Although modest bivariate associations were detected with educational attainment (r = .17) and body mass index (r = −.17), almost all regression-adjusted coefficients were nonsignificant. No clear pattern of moderation was detected between delay of gratification and either socioeconomic status or sex. Results indicate that Marshmallow Test performance does not reliably predict adult outcomes.
I guess the question is whether the covariates that were adjusted for in the regression are true confounders and not, say, something caused by ability to delay gratification.
by oglop on 8/2/24, 9:19 PM
Yeah, a fucking marshmallow won’t do much to predict you future. Family wealth does.
by veggieroll on 8/2/24, 5:33 PM
I don't like marshmellows.
by dfedbeef on 8/2/24, 6:35 PM
That test was already broken a decade ago by Kidd. The socioeconomic part of it is BS and has been known to be for a while.
by nineplay on 8/2/24, 5:22 PM
I was talking about this a few weeks ago and realized I would eat the damn marshmallow. Researchers do not act in good faith. Maybe they're testing me for delayed gratification. Maybe they're measuring my anxiety levels as I wait for someone to come back with a promised reward. Maybe they want to know how angry I'd get if they come back and said they were out of marshmallows - or come back and flat out ate the marshmallow in front of me. A lot of researchers would happily trick me into thinking I was killing someone if they thought they could get away with it.
Its the truth that demolishes all the hand-waving about the marshmallow test - it relies on the subject's trust of the person running the experiment. I wouldn't trust them, why should anyone else?
When evaluated that way - particularly when testing on children - the outcome is painfully predictable.
- Children who have adults in life that they trust have better outcomes.
- Children who do not have adults in their lives who they trust have worse outcomes.
by vvpan on 8/2/24, 5:18 PM
Yet another study that "explains it" turns out to be false. Good.
by poindontcare on 8/2/24, 7:02 PM
hahahaha!
by dudeinjapan on 8/2/24, 5:07 PM
They only measured the subjects' "adult" life outcomes at age 26. Perhaps the researchers were rushing to publish and unwilling to wait long enough for the effect to replicate.