from Hacker News

Apple Vision predictive bio-feedback

by rmm on 6/6/23, 9:09 AM with 186 comments

• by sterlingcrispin on 6/6/23, 8:18 PM

  Hey I appreciate the general lack of cruelty so far in the comments, sometimes HN can be pretty rough.

  The weird phrasing in the tweet is because I can't talk about any of the work I did due to NDA's besides my job description and those patents. But I had a pretty unusual role all things considered. I was also trying to clarify in the tweet that these aren't meant to be interpreted as unannounced existing features or capabilities of the product. But rather, research I did and contributed to, that was made public through patents.

  There's a ton of interesting preexisting research literature about neurotechnology research for VR. Both in terms of using it for biofeedback, and using it as a device just for studying the brain. If that sounds interesting browse through what's out there via google scholar.

  Also I said I spent 10% of my life working on it, realizing that was a goofy way of putting it. But it's a standout thing in my life and it was mostly a self reflection on what a trip being alive is, not trying to brag about it, mostly just sharing what I did. It's an unusual product and I had an unusual role

  Anyway it's a cool device and hopefully people enjoy it.

• by makeitdouble on 6/6/23, 9:54 AM

  This whole tweet had the same tone as when pop science crime investigation TV explains a criminal can be identified through its gait alone.

  It's always magic, except when the predictive model doesn't work and there is no corrective action possible on the user side.

  We're talking about physical reactions, so that means it's supposed to be the same through gender, age, race, health conditions from the algorithm's point of view...That's a lot of variables, and I wonder if Apple have analyzed this beyond the US population, assuming the device will be sold internationally at some point.

  Apple seems really confident about this as they are foregoing controllers on the default interfaces, but that also reminds me on how they were rumored to make a car without a driving wheel...fingers crossed, I guess.

• by vincnetas on 6/6/23, 4:32 PM

  "All of these details are publicly available in patents, and were carefully written to not leak anything."

  This part makes me sad about patents and original idea behind them. This actually is the opposite of original idea why people came up with the idea of patenting something. To spread the knowledge and to award inventors. Not to stifle the competition.

• by noindiecred on 6/6/23, 11:23 AM

  Oh so now the hot new thing is a computing device that actively tries to guess its user’s emotional state (using various methods, including introducing new stimuli) and then then CHANGES ITS BEHAVIOR based on the results!

  The Free Software people were right - a device this intimate needs to have publicly available, verifiable source code.

• by kasperni on 6/6/23, 9:38 AM

  > So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.

  I don't really know how I feel about this...

• by seydor on 6/6/23, 10:54 AM

  I know we all hate each other but is this like a good direction for the future? Directing all our emotions to silicon chips? Maybe there is other things we should be building.

  Emotion recognition is an old field but (apart from autism research) the only application i can think of is emotion manipulation. How does one wake up in the morning and decide to work on consumer emotion recognition

• by kossTKR on 6/6/23, 10:52 AM

  I can't really grasp how enormous this project is.

  5000 patents filed from Apple is crazy, and looking at the thing with all of these new both pieces of hardware and concepts, wow.

  How many people have been working on this, and for how long? The other threads here seems to have missed just how "huge" a projects this is.

  Is this also the lead up to the Apple car, or some other ecosystem?

• by Bukhmanizer on 6/6/23, 4:59 PM

  This is a pretty shockingly bad statement. Not only is it a complete exaggeration, it’s not even a good exaggeration. Saying “I worked on mind-reading features” for Apple’s new VR headset is just creepy.

• by thorum on 6/6/23, 8:11 PM

  Public patents or not, can't imagine Apple is going to be thrilled with this guy for announcing to the world that their new product is secretly reading everyone's minds.

• by ugh123 on 6/6/23, 4:20 PM

  next level humble-bragging. in post and his replies

• by Traubenfuchs on 6/6/23, 10:17 AM

  I wonder if him not having worked on Apple Vision for the last >1.5 years means they distanced themselves from his "vision, strategy and direction" or even gave up on it.

• by prashp on 6/6/23, 6:58 PM

  Maybe he was fired in 2021 for being too modest.

• by thih9 on 6/6/23, 9:57 AM

  It looks like this is mostly about patents. Apple has a history of doing lots of hardware and software research, not all of that gets to be part of an actual release.

  Is anything indicating that solutions mentioned in the post would actually become part of Apple Vision?

• by lrvick on 6/6/23, 11:56 AM

  I own 0 Apple products and will never give them money, but this post really helped reaffirm my desire to not have Apple products anywhere near me.

• by exitb on 6/6/23, 8:32 PM

  Soo… is this real? Did those features really make it into the product as of now? I don’t recall any of these details being mentioned.

  > measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.

  The headset does not appear to track most of these things.

• by BasedAnon on 6/6/23, 2:32 PM

  Right... so this thing hooked up to a centralized AI, deciding what people see when, how often they see it, what they can express and how they feel... Do people really want to live in the matrix? I have a visceral reaction to this idea

• by AlexanderTheGr8 on 6/7/23, 2:45 AM

  Sorry for hijacking this thread for a question about personal curiosity but does anyone know of an affordable way of measuring pupil distance?

  I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible.

  Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...

• by euroderf on 6/6/23, 8:25 PM

  > Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.

  Is this like how in the old days the movie theater flashed a single film frame of BUY POPCORN YUM.

• by dahwolf on 6/6/23, 10:37 PM

  So Apple can tell how you bio-physically respond to dynamic input. No worries though, we'll show a blend of an Apple logo and lock, which means all is fine. Go back to sleep, but we'll let you know if it was a proper sleep.

• by dangus on 6/6/23, 12:49 PM

  This post: “I’m a mercenary helping to make the thought police a reality, hire me.”

• by mhh__ on 6/6/23, 10:57 AM

  Valve have been interested into the biological side of VR and BCI for a while too

• by DoreenMichele on 6/6/23, 4:53 PM

  Social stuff is best approached from a statistical confidence model that allows for "This might be the one in ten times my assumption is wrong."

  I wonder if/how they incorporate that.

  Because a lot of social drama comes from humans betting on the statistically likely thing in social situations and it being wrong rather than checking or hedging their bet. ("I shall assume the white male is in charge. Oops. Now I've offended the woman or person of color who is actually in charge.")

• by stevebmark on 6/6/23, 11:46 PM

  Demystifying this concept: The mechanism is almost certainly just measuring pupil dilation, and maybe eye focus, if that's something you can track with an eye facing camera.

  Source: Thinking Fast and Slow, where the initial inspiration for the book was an experiment noting that human's pupils reflexively dilate when they engage their "system two," aka the "slow brain," metaphorical part of their mind.

• by lumb63 on 6/6/23, 8:21 PM

  This begs the question: what is Apple’s real play with Apple Vision? It sounds like there’s a real possibility that, even at a $3500 price point for hardware, the consumer is still the product. What else can their system determine about someone’s brain? What is going to be done with that data?

• by joe5150 on 6/6/23, 6:16 PM

  Not hard to imagine how tools to predict and manipulate your emotions might be useful in a device that even in the wildest dreams of its developers is mainly designed to provide an immersive environment for taking Teams calls and working on spreadsheets.

• by kcrwfrd_ on 6/6/23, 11:37 PM

  I’m currently undergoing a course of neurofeedback therapy… the biofeedback tech in the Apple Vision sounds like it could have some pretty cool applications for such things.

• by fsloth on 6/6/23, 9:46 AM

  The perfect Skinner box.

• by renewiltord on 6/6/23, 4:57 PM

  This is pretty cool. Eager to see what it feels like in reality. The risk with prediction is misprediction. Causes a lot of frustration.

• by serf on 6/6/23, 9:51 PM

  this whole release feels extraordinarily 'Apple' , to me.

  Apple takes an idea and staples as many 'side-grade' style functionalities to it as possible in order to sell it , rather than just offering a really well-performing product within the niche.

  The iPhone was a remarkably bad phone -- it was more awkward to hold than any other phone at the time (now they're ALL awkward), and the reception was terrible. The provider it was locked to at the time was one of the worst in North America at launch; but the iPhone had features that people went nuts for which drove sales beyond the phone offering.

  I feel like this VR headset will be similar.

  It's a mobile device with a 2 hour battery life -- something that everyone in the world has been saying would make a device dead on arrival for the mobile market; but it has all of these little side-grade features that will attract purchasers that are interested in the Apple product itself but not necessarily VR.

  As a consumer i'm glad -- I may have never been a big iPhone fan, but the Android reactionary effort produced so much value that I can hope that this field will see something similar once it proves the market feasibility. I can't wait for my 8 hour/8k res/quarter-priced white-label VR headset clone in a few years.

• by vezuchyy on 6/6/23, 5:09 PM

  Wait, how did I miss that it is now possible to tweet full-size essays?

• by MagicMoonlight on 6/7/23, 2:24 AM

  Lmao that sales pitch trying desperately to get a job

• by riffic on 6/6/23, 4:55 PM

  tweet should have been a blog post.

• by justinator on 6/6/23, 4:18 PM

  I went to art school with this guy.

• by nbzso on 6/6/23, 5:08 PM

  I really miss Steve Jobs.:)