from Hacker News

RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning (2023)

by bemmu on 2/27/25, 9:41 AM with 53 comments

• by pama on 2/27/25, 12:48 PM

  This was a nice demonstration of letting these robotic hands learn to play a keyboard. The technique is limited due to constraints of the hands, and is closer to parts of the technique of early keyboard instruments like an organ or a harpsichord, rather than that of a modern grand piano, which requires a lot more control of the body core, shoulders, elbows, arms, and wrists with the fingers doing as minimal a motion as possible. I suppose a similar algorithm learning to play on a grand piano using a full humanoid body could learn a technique that would be exciting to analyze.

• by kbouck on 2/27/25, 11:47 AM

  Any way we can get the Westworld intro song into the demo list?

• by ericye16 on 2/27/25, 6:33 PM

  This was the basis of a project I did for my deep reinforcement learning class!

  https://ericye16.com/stanford-cs224r

  We were able to make some improvements by tuning how the reward is distributed and also by first pretraining the agent on scales before fine-tuning them on the final pieces.

  Thanks to Kevin Zakka for helping us get started with the RL environment!

• by kittikitti on 2/27/25, 11:37 PM

  > UC Berkeley, Google DeepMind, Stanford University

  I've personally experienced how research around this time was being shut down because of AI doomerism. People were getting laid off because of it. It's clear to me that these institutions actively spread AI doomerism so that they have full control over it. They actively called for a stop in AI research so that their personal labs can leap forward ahead. It was a little too on the nose for Big Tech, but they don't understand nuance.

• by criddell on 2/27/25, 12:34 PM

  Is this actually buildable? I'd like to hear it on a real piano because (IMHO) the one in the videos sounds bad.

  I'd like to also hear how loud the mechanical noise of the machine playing the piano would be. Does the left hand work harder with the heavier keys? What would the hands be mounted to?

• by rossant on 2/27/25, 12:45 PM

  Nice work! And great interactive 3d application. My 6yo had a lot of fun annoying the robot while it's playing by forcefully moving its hands around.

• by beardyw on 2/27/25, 1:11 PM

  All the right notes but not necessarily at the right time.

• by Tycho on 2/27/25, 2:53 PM

  Has anyone tried to get a robot to do oil paintings?

• by fedeb95 on 2/27/25, 11:05 AM

  waiting for Robot Devil.

• by davidanekstein on 2/27/25, 12:24 PM

  Funny enough I’m listening to Rhapsody in Blue while browsing HN. I’d like to see it do that for 17 minutes.

• by marbro on 3/1/25, 12:10 AM

  How does this research help us win naval battles? Should Elon and his DOGE friends cut it?

• by gtirloni on 2/27/25, 2:26 PM

  Imediatelly reminded of the anonymous quote: "I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes."

  I understand it's a bad argument because 1) people may need assistance to do art/writing and 2) the advancements gained from teaching AI to do art can be applied to other non-artistic endeavors (e.g. piano AI could be really good to operate machines with lots of buttons and no computerized interface).

  However, the cost cutting side of the argument is the one that bothers me because companies/people WILL use AI like that in place of actual humans because they're likely to be cheaper in the future. So that pianist or musician playing in a local restaurant can be sure their job will be automated away by a subpar AI and real humans will be relegated to very expensive locations (an extension of replacing humans with recorded music, in a way).

  My pessismist side thinks greed will be the downfall of humanity.

• by iwontberude on 2/28/25, 4:14 AM

  Wasted opportunity to not play the Westworld Theme

• by wbakst on 2/27/25, 9:06 PM

  the fact that the failure modes come from limitations of the robotic hands is super interesting

• by torusle on 2/27/25, 4:08 PM

  Honestly,

  this is really bad. It might be a breakthrough of what you are doing, but when I listen to the output all of the timing and phrasing is aweful.

• by amelius on 2/27/25, 12:16 PM

  I mean, keys are always in the place where you expect them to be, so what is the challenge here? (Yes, too lazy to read the article).

• by pacificmaelstrm on 2/27/25, 2:46 PM

  "Player Piano"

• by PatronBernard on 2/27/25, 12:56 PM

  The thing needs a teacher, terrible technique! /s