from Hacker News

Robust autonomy emerges from self-play

by reqo on 2/7/25, 2:44 AM with 62 comments

• by markisus on 2/7/25, 5:44 PM

  Some interesting points from this paper:

  - All simulated agents use the same neural net with the same weights, albeit with randomized rewards and conditioning vector to allow them to behave as different types of vehicles with different types of aggressiveness. This is like driving in a world where everyone is different copies of you, but some of your copies are in rush while others are patient. This allows backprop to optimize for a sort of global utility across the entire population.

  - There is no modeling of occlusion effects. Instead, agents are given the state of nearby agents, but corrupted by random noise. In the real world, occluded nearby agents can be extremely close (think about a child running out from behind a parked car). The paper comments on this.

  > Both Waymax and nuPlan construct observations, maps, and other actors with auto-labeling tools from realworld perception data. This brings occlusion, incorrect or missing traffic-light states, and obstacles revealed at the last moment. Despite the minimalistic noise modeling in GIGAFLOW, the GIGAFLOW policy generalizes zero-shot to these conditions.

  - The resulting policy simulates agents that are human-like, even though the system has never seen humans drive. This is a great result when one considers other reinforcement learning projects produce extremely high performance agents that humans would consider to be abusive or pathological.

• by nine_k on 2/7/25, 3:53 AM

  Can there be "smart toys" for models that help them self-improve in a particularly efficient way?

• by seaucre on 2/7/25, 3:49 PM

  This is interesting, and I have always thought this approach worth exploring given the "bitter lesson" in other ML domains, but I think we should be skeptical until we see such models deployed and operating effectively on real-world vehicles.

• by dhbradshaw on 2/7/25, 4:04 PM

  Interesting to see this coming out of Apple

• by mitthrowaway2 on 2/7/25, 4:55 AM

  Something about dreams that fascinates me is that I usually am genuinely surprised by events that occur in dreams. I interact with other characters whose motivation I cannot understand and whose actions I cannot fully anticipate. It feels like there's a foreign entity acting as DM.

  This isn't fake surprise. Sometimes I'll wake up and think, "who on earth were those guys and what were they trying to do? And yet their actions make sense..." or, "who came up with that punchline? It's legitimately funny and I never saw it coming, so it can't have been me..."

  And yet I know it's all being generated by my own brain somehow. Through some kind of privileged access level.

  And then I think about the bicameral brain structure. Does our brain have two halves so that it can function in a self-play training mode during sleep? Are each halves of my brain experiencing the same dream from opposite points of view?

  Apologies for the tangent; this is almost totally unrelated to the article and probably something well known to neuroscience for decades. But still, it fascinates me, and the more we learn about the effectiveness of self-play in AI, the more I wonder.

• by linux_devil on 2/7/25, 10:48 PM

  Maybe not directly related , I find genertic algorithms and other optimisation algorithms such as Ant Colony Optimisation algorithms intersecting with this approach of self-play and leading to robust autonomy.

• by The28thDuck on 2/7/25, 4:48 AM

  The concept of being able to simulate 42 years of “experience” in one hour seems so foreign to me. Something about it creeps me out.

• by dang on 2/7/25, 7:35 AM

  [stub for offtopicness]

• by surume on 2/7/25, 12:12 PM

  [flagged]