by firloop on 6/6/23, 9:55 PM with 688 comments
by jb1991 on 6/7/23, 6:43 AM
And these 3D spacial moment recordings, imagine children growing up in a house where when something nice happens, the parent rushes to put on goggles and stare at them through them, their little virtual eyes displayed on the outside, it’s frankly creepy to me.
by dvt on 6/6/23, 11:18 PM
And I'd say that 80% of its failure can be attributed to bad software. It's buggy, I constantly have to re-calibrate my work/play area, the hand tracking is janky, and even though it supports (incredible) eye tracking, almost nothing takes actual advantage of the hardware.
It's insane that a billion-dollar company like Meta actually felt proud to release such a steaming pile of trash. Do they seriously expect VR enthusiasts to build an entire operating system for them? Much like the reports we're getting about the Vision Pro, the eye tracking in the Quest Pro feels like magic, but nothing uses it—basically it's irrelevant to navigating the operating system and barely any games use foveated rendering. It's infuriating.
If I was Zuck, I'd fire all my product managers. I say good for Apple. I'll probably be selling my Quest Pro and buying the Vision Pro.
by JohnBooty on 6/6/23, 10:45 PM
But it does put an enormous amount of pressure
on the eye tracking. As far as I can tell so
far, the role of precise 2D control has been
shifted to the eyes.
I'm extremely curious to know how Vision Pro deals with this. One certainly hopes there's some kind of "single eye" mode; certainly seems possible with relatively small effort and the % of the population who'd benefit seems fairly significant.
Eye tracking most certainly sounds like the way to go, relative to hand-waving.
The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
by smugma on 6/7/23, 1:24 AM
Andy’s thoughts were infinitely deeper than anything else I’ve read about the device+interface so far. In particular, I liked his observations that so far it’s mostly a 2D plane, with exceptions like the Breathe app and a few demos, which might just be good for demos.
Seeing the heart in 3D reminded a lot of what we saw 10 years ago around what book publishers were trying to do with the iPad. Cool demos but limited real world use.
Here was one particularly thoughtful section:
“unlike the physical version, a virtual infospace could contend with much more material than could actually fit in my field of view, because the computational medium affords dynamic filtering, searching, and navigation interactions (see Softspace for one attempt). And you could swap between persistent room-scale infospaces for different projects. I suspect that visionOS’s windowing system is not at all up to this task.“
by lukevp on 6/7/23, 1:04 AM
by BoppreH on 6/6/23, 11:27 PM
- Something whimsical to inspire developers. Give me files represented as physical blocks that I can pile, or a task manager that shows processes as plants around me. Close apps by scrunching their screen like paper. A globe with my geotagged pictures.
- A game, any game. Beat Saber was the killer app for me to get my Vive, and Valve already spent tens of millions of dollars to create a triple-A VR game. Neither seem compatible with the Vision Pro input methods. Apple could at least play their strengths, like an AR hide-and-seek, or a horror game with eye tracking.
- Content creation or professional work. The last thing we need is another passive device to watch 3D TikTok. Show someone customizing their "home screen" environment to look like a fantasy potion shop; or a mechanic looking at a 20x magnified broken part; or an SRE watching their kubernetes cluster as a floating 3D graph; or a VFX artist in the scene scrubbing forward and backward to adjust an effect.
It feels like inventing the first smartphone, camera apps and all, but it doesn't make calls and the only input is tapping on icons.
by jillesvangurp on 6/7/23, 6:32 AM
- leverage existing content means they won't have an empty room problem or a big "now what?!" moment for users after the novelty wears off. Without Steam, most existing VR platforms would be completely pointless. So far VR is for games and most of those are published on Steam. That's because most VR hardware vendors suck at software and end up outsourcing that to game studios. Meta included. Despite their ambitions, their goggles are mainly devices for running games not developed by Meta.
- a focus on the living room experience with a high end movie theater experience running, again, off existing content. Genius move because people already spend lots of money on home theaters. Some people buy 10K$ plasma screens even. This market is very real. Also, involving Disney with their huge back catalog of completely unmonetized 3D movies ... so obvious.
- Extend the highly successful IOS ecosystem to the new experience. They have tens of millions of apps already. And they'll work fine in AR. Why not do that?
The strategy is about content. It's the right strategy. The first generation hardware is of course amazing too and it will bootstrap a new generation of application developers that will be using Apple SDKs and tools to target all this with new content. But to bootstrap the ecosystem they need an audience.
Here too the strategy is genius: SDKs are based on things developers already use. They just co-announced a push into gaming for Mac along with some convenient porting kits for developers. That was just a footnote buried deep into the mac related announcements. But of course this means more content coming to AR as well. All that lovely content currently available via Steam.
Top to bottom the strategy is about compelling content. I think it's going to work.
This is not the final answer to AR but the opening salvo in a decades long push to completely own this space. Step 0 is to get millions of these things in the market with enough content to get people hooked and keep them consuming content. The rest will come later. It's appropriately radical, pragmatic, and conservative at the same time.
by gnicholas on 6/6/23, 10:17 PM
I actually see this as a bit less surprising. After all, if you change the hardware in a big way, and you change the software in a big way, users will have a harder time adjusting to the new platform. Instead, they're making a big leap on the hardware side, keeping legacy apps and concepts, and then will presumably iterate to more 'native' experiences that were previously impossible/unimaginable/unintuitive.
by cobertos on 6/7/23, 5:14 AM
* Fingers together gesture clicking.
* Voice activated menu navigation. It was glitchy/I never used it though. In 2023 these sorts of systems are much better though.
* No controllers. It was all gesture based. Opening the start menu required your hand upturned, fingers together then outstretch. Kind of like an "open" gesture.
* "Pointing" based on head looking vector, which was annoying.
* Spatial anchors and being able to remember past spaces and how you used them. There was a whole set of SDK APIs for the spatial stuff built into Win 10
The Vision Pro iterates on some of these. Eye tracking for pointing and more cameras for tracking hand pose for clicking, to reduce the annoyance/strain. IMO all mandatory for long term usage (I frequently got motion sick with the HoloLens and the 3 months of debugging/developing on it were a challenge).
Maybe if Microsoft had switched to more commonplace display technology and continued to iterate on their product instead of let it languish, they could have had a solid competitor to this, if not maybe be first to the market.
by zmmmmm on 6/6/23, 10:44 PM
I'm disappointed even though it's entirely predictable that VisionOS is built on the iOS/iPadOS foundation rather than OSX. I guess we'll see how "walled in" it is but it's hard to see any reason Apple isn't going to be just as constraining and controlling about what happens on this OS as they are on iOS, if not more so. Which ultimately means I'll be very reluctant to ever adopt it in any meaningful way as my primary computing device.
by skydhash on 6/7/23, 1:51 AM
I'd prefer an OS that spans multiple devices. Something how computers behave in “The Expanse” where someone can flick stuff from their mobile device to another system for a better workspace. If I'm typing this on my workstation, but want to switch environment and bring something light like a tablet or just my phone, I could send the tab and it would retain its state. The closest to something like this is a server-client architecture like Logitech Media Server. A paradigm like this would be more useful than immersion.
This is also why I like Vim. Buffer is a separate concept from File, adding an extra flexibility – in terms of layout – to how I edit to complete my current task. I want a similar flexibility with applications, at least lightweight – state wise – ones.
CloudKit and Handoff are close, but very brittle. Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless. Another good example is handoff between the HomePod and the iPhone.
by armchairhacker on 6/6/23, 11:37 PM
Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper.
Imagine playing a video game but the environment wraps around you. Imagine watching movies in a virtual theater with a simulated 100-inch TV (one of Apple's demos). Even reading and browsing the web can be improved with an ambient environment and extra space for more stuff.
Is it worth $3500? For end-users probably not, but if it genuinely makes professionals and hobbyists substantially more productive, it will be worth $3500 or even way more. How much money would you spend to write, code, create faster?
Of course, this assumes the VR actually performs, and whether a bigger screen and immersion actually makes people work faster. As of now it seems VR is still a gimmick which impresses people at first, but doesn't provide much outside of niche experiences; which, if this holds for the Vision Pro, makes it very much not worth the price.
by david_van_loon on 6/6/23, 11:32 PM
by layer8 on 6/7/23, 12:00 AM
by gnicholas on 6/6/23, 10:21 PM
by mcintyre1994 on 6/7/23, 5:33 AM
by fio_ini on 6/7/23, 5:49 PM
by whywhywhywhy on 6/7/23, 12:13 PM
Can’t believe after the failed attempts to kill files on iPhone and iPad they’ve learned nothing and still think we can pretend a professional work tool can exist without a proper file system.
Not saying it should have desktop icons strewn across your living room, just that the “files siloed in apps transferring via share buttons” iPad model is objectively productivity hell and puts mental load menu fumbling where MacOS is instinctive drag and drop.
by xivusr on 6/6/23, 10:15 PM
by ftxbro on 6/7/23, 8:48 AM
I don't understand this. It's things they made them sign? I saw another article saying they make them sign a promise to not show they are wearing the headset, allegedly because they might look like huge dorks and cause brand risk.
by elif on 6/7/23, 9:32 AM
As long as it doesn't get sweaty or nauseating after 30 minutes, I would still be content with it as a glorified monitor, not needing new UI paradigms or gesture controls.
Distraction-free huge monitor computing anywhere in the world is a huge advantage over laptops imo.
by coryfklein on 6/7/23, 3:49 PM
Why would anyone want text fully aligned to the left of the browser window?! At least it's distinctive, I don't think I've ever seen that on any other website in the last 15 years.
by 8jef on 6/7/23, 2:08 PM
Now I just have to wait for a Linux compatible version.
Because there's no way I'm gonna use any Apple software as if it was twenty five years ago, before iTunes insanities and whatever happened since. NFW.
by cudgy on 6/7/23, 10:29 PM
This is why I think this device will be different than its predecessors. There are practical reasons to buy it … assuming you can use it without getting sick or strained.
by robwwilliams on 6/7/23, 3:14 AM
Gary has a great TED talk (Mar 2010) on this radical but intuitive spatial sorting interface. Foveation and hand movements are perfect user links.
https://www.youtube.com/watch?v=LT_x9s67yWA
Combined with a Vision Pro—OMG.
by sebkomianos on 6/7/23, 12:47 AM
by dboreham on 6/7/23, 12:03 PM
That said, the same thing was true about video phones, to the point that everyone assumed humans do not need to see each other on a phone call. Then we had a pandemic and things changed.
by notnotjake on 6/7/23, 4:30 AM
by wouldbecouldbe on 6/7/23, 12:20 PM
by tromp on 6/7/23, 10:14 AM
Figuring out how you're going to get mobile data access in a foreign country without being charged through the nose does feel somewhat stressful...
by pflenker on 6/7/23, 7:04 AM
by brandall10 on 6/7/23, 4:57 AM
I thought at this stage, it would be fairly foolproof to use that at the very least to simulate something like a click, no?
by amelius on 6/7/23, 12:54 PM
So I take it that you can basically leave your laptop home and take your Vision visor with you and work from a virtual environment?
by uxp100 on 6/7/23, 12:46 PM
by AlexanderTheGr8 on 6/7/23, 2:37 AM
I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible.
Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...
by nullandvoid on 6/7/23, 6:14 AM
I wonder what happens in games though. Will this not essentially be aimbot, looking at the target and then shoot?
by dav_Oz on 6/7/23, 2:05 PM
The basic observation: It takes very little effort and exposure to remember places (spatial information) in contrast to taking in linear heavy processed information from writing and numbers (e.g. "facts").
Combine these two and you have a powerful way navigating through the information space itself.
If you look at it, we are confined to rectangular boxes, fancy interactive books enhanced by moving pictures, audio and some very limited tactile feedback but nevertheless bound by the superstructure of a static tiny detail in our overall spatial awareness.
Of course we are now so used to the endless iterations of rectangular boxes that we are extraordinarily proficient at extracting a lot of richness/information from that: in scrolling, clicking, typing, imagining ... I'm reminded by the scene of "The Matrix" where Cipher let Neo glimpse the Matrix code on his screens: "You get used to it, I don't even see the code. All I see is blond, brunette, redhead."[1]
IIRC once a 50s western movie was shown to a rural "test" public from the Eurasian steppe some of who saw a moving picture for the very first time. The reactions were mostly enthusiastic during slow "panorama shots" but the audience reacted very agitated in "close-ups" or "medium shots" where e.g. in the heat of the action the horse's legs were "cut off".
We take it as given that we learn the language of movies and find them at times 'realistic' and 'immersive' when in fact contrasted to 'reality' itself qbit by qbit they are basically - with some rounding error - as highly processed as books and thereby rely heavily on our imagination filling the gaps.
While we are on the subject of of naïve visions, another area which isn't appreciated enough imho is the immense dexterity of our hands as with "vision processing" the devices themselves are glued on our dexterity of "exact" and "repetitive" moves, there is too little wiggle room for organic experimenting, refining and expression like one would naturally do for a musical instrument.
by sw104 on 6/7/23, 11:34 AM
Maybe this is a better time than ever to start a new opticians.
by gampleman on 6/7/23, 8:34 AM
It seems like the last few years have generally had a technological backlash with worries about both invasive spying and surveillance capitalism on the one hand and mental health issues (especially in teens) sprouting on the other.
So it seems strange to react to all that by offering a product that glues a computer straight on your eyeballs. Now of course Meta has already been going down that road, but let's just say that company doesn't have much to loose in the reputation department.
To be fair, Apple seems to have made a few interesting design choices. They seem to have gone out of their way to create an illusion of two way transparency to their headset making casual human communication at least theoretically possible (although how much conversation you can have with someone actually wearing these due to distraction is an open question - if it's like trying to talk to someone on their phone that it will be at best half their attention).
by stavros on 6/6/23, 11:17 PM
by russellbeattie on 6/6/23, 10:41 PM
I've been researching eye tracking for my own project for the past year. I have a Tobii eye tracker which is probably the best eye tracking device for consumers currently (or the only one really). It's much more accurate than trying to repurpose a webcam.
The problem with eye tracking in general is what's called the "midas touch" problem. Everything you look at is potentially a target. If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it. [1]
Additionally, our eyes are constantly making small movements call saccades [2]. If you track eye movement perfectly, the target will wobble all over the screen like mad. The ways to alleviate this are by expanding the target visually so that the small movements are contained within a "bubble" or by delaying the targeting slightly so the movements can be smoothed out. But this naturally causes inaccuracy and latency. [3] Even then, you can easily get a headache from the effort of trying to fixate your eyes on a small target (trust me). Though Apple is making an effort to predict eye movements to give the user the impression of lower latency and improve accuracy, it's an imperfect solution. Simply put, gazing as an interface will always suffer from latency and unnatural physical effort. Until computers can read our minds, that isn't going to change.
Apple decided to incorporate desktop and mobile apps into the device, so it seems this was really their only choice, as they need the equivalent of a pointer or finger to activate on-screen elements. They could do this with hand tracking, but then there's the issue of accuracy as well as clicking, tapping, dragging or swiping - plus the effort of holding your arms up for extended periods.
I think it's odd that they decided that voice should not be part of the UI. My preference would be hand tracking a virtual mouse/trackpad (smaller and more familiar movements) plus a simple, "tap" or "swipe" spoken aloud, with the current system for "quiet" operation. But Apple is Apple, and they insist on one way to do things. They have a video of using Safari and it doesn't look particularly efficient/practical to me [4].
But who knows - I haven't tried it yet, maybe Apple's engineers nailed it. I have my doubts.
1. https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-...
2. https://en.m.wikipedia.org/wiki/Saccade
3. https://help.tobii.com/hc/en-us/articles/210245345-How-to-se...
by nerdbert on 6/7/23, 9:08 PM
by andsoitis on 6/7/23, 3:15 AM
? =! Vision Pro
by TheRealPomax on 6/6/23, 11:35 PM
by ganesh7 on 6/7/23, 10:41 AM