Although we still have a long way to go, we are definitely closer to our goal than ever before.
The number of people working in the computer industry is huge and growing. These people use personal computers, laptops or employer-provided devices. However, productivity is often limited by the number of screens available, their cost, the inconvenience of travel, privacy concerns (when the screen is visible to others), inadequate screen resolution and the restriction of content to a two-dimensional view on a flat screen.
Mixed reality goggles seem to solve many of these problems. Apple's Vision Pro, for example, offers crystal clear image quality with high resolution across all screens. Both Vision Pro and Quest 3 allow for multiple 'floating' screens, positioned and scaled to your needs (with some limitations).
With these goggles you have complete privacy, as only you can see the floating screens. You can also travel with them wherever you go. In addition, you can isolate yourself in beautiful virtual environments or return to reality using the cameras in both goggles.
Given these benefits, why does working on VR/AR goggles still seem... so strange? Well, I think I already know the reason. I spent a week working on Meta Quest 3, using it for everything I do on a daily basis. Luckily for me, Meta released the v67 update around the same time I decided to do this. The v67 update allows users to detach virtual windows from the 'dock' they were previously attached to, and place them anywhere in space. This makes the Quest 3 experience more akin to Apple Vision Pro, with the additional benefits offered by Quest 3. Let's dive into the details.
Working on Meta Quest 3 - My experience
To work on Quest 3, I needed a few things. A computer, of course, but more than that. My work involves managing social media accounts, writing articles and editing photos and videos. Fortunately, Quest 3 is powerful enough to handle the first task on its own, without being connected to a computer. However, a connection to a computer is required for photo and video editing.
With the v67 update, which allows you to essentially work at the operating system level without using an app, I decided to buy a dedicated Logitech keyboard and Bluetooth mouse for my Quest 3. When I needed to work on my PC to edit video or photos, I would run a remote display on my PC and in the goggles, connecting my PC to my Quest 3. This setup is limited to one screen, but you can have other screens in the goggles, such as a web browser, open alongside the remote display.
Some apps, such as Immersed, allow you to use multiple screens mirroring the PC's, but in my testing this proved to be a poor experience - it was hard not to notice a lot of lag, and the resolution of the virtual screens left a lot to be desired. Despite having a Wi-Fi router 6 and trying to connect the Quest 3 to the PC for better data transfer, it didn't help much. So I stuck with the remote display from Meta and got to work.
Let's start with the positives. I started writing the article using the goggles, and almost all of the work was done in the Meta Quest browser with multiple windows open. It's worth noting that, unlike Quest 3, Apple Vision Pro has built-in apps, so we're not limited to just the browser. Linking all my accounts was a little cumbersome, but understandable as Meta doesn't have an integrated ecosystem like Apple, where one login gives you access to almost everything.
Initially, my experience of writing an article and conducting 'research' was quite positive. I was able to create up to six windows - three floating wherever I wanted and three attached to the dock - and I made good use of this feature. Although I mainly used three windows, the increase in productivity was noticeable. I could easily run my word processor, Twitter, analytics, Google and anything else I needed right in front of me.
The keyboard and mouse setup is quite comfortable. Despite a few problems, which I'll mention later, they are generally useful and even enjoyable to use. It is a unique experience to see the mouse cursor hovering in the actual space between windows. I was also able to easily move and resize the windows using the mouse.
The keyboard worked well and even had its own (albeit imperfect) tracking system that allowed the use of a 'passthrough portal' when working in VR.
Another big advantage is the ability to work anywhere with just the goggles, mouse, keyboard and Wi-Fi connection. On sunny days, I often enjoyed working outdoors on my terrace (trying not to damage the lenses or screens, of course). I enjoyed using the Lakeside Park Met's virtual environment, where I could play soothing music and feel the breeze blowing as I gazed at the picturesque view of mountains and forests. This feature is fantastic for focusing on work without the distractions of reality and for getting a little creative with stunning landscapes.
Problems
My experience of writing articles and managing social media using Goggles was decent. While it was possible to complete tasks, I have to admit that it was not a better experience compared to working on a computer. The ability to have multiple screens, immerse yourself in a beautiful environment and work from virtually anywhere are noticeable advantages. However, these benefits are overshadowed by several problems that reduce their overall impact. Let's look at the main problems I encountered.
Here is a list of key areas for improvement, which I will discuss in detail in the article:
- Convenience
- Resolution
- Processing power
- System and integration - UI/UX
Convenience
"Comfort" is an umbrella term here - it encompasses many factors, including comfort, weight, ease of use and the desire to use the goggles. Let's look at these aspects.
For the Questa 3, I use the Kiwi Design head strap and AMVR interface to make the goggles more comfortable. They significantly improve comfort and balance. Without these accessories, the standard head strap would make working with the goggles quite uncomfortable within an hour.
However, even with the comfort accessories, I can still feel the goggles on my head. I can wear them for about six hours without much discomfort, but the urge to take them off for a few minutes increases over time. Additionally, while the accessories improve balance, they also increase the weight of the goggles, so the weight is still noticeable.
Ease of use and willingness to use goggles are as important as weight and comfort, and often go hand in hand. Consider how many people would be reluctant to wear even the most comfortable and lightweight goggles if they ruin their hair or have a poor interface and user experience. If you're bald, you obviously don't have this problem, but many people don't want to ruin their hair after using goggles and then leaving the house.
Resolution
Image quality remains a significant issue with current goggles. While Apple Vision Pro has made significant strides in this area, it comes with a hefty $3,500 price tag. In addition, there are rumoursthat even the more 'affordable' Apple Vision goggles may have a reduced resolution to cut costs.
I was working on the Quest 3, not the Apple Vision Pro, so my opinions are based on the resolution of these goggles. Quest 3 offers a resolution of 2064×2208 per eye, which is about a 30% improvement over Quest 2. The new pancake lenses also provide a noticeable improvement over the previous generation's Fresnel lenses. Nevertheless, we are still not quite where we want to be.
I was able to read text without having to move my head much, thanks to the pancake lenses providing an almost crystal clear image and a decent field of view. So, what's the problem?
The problem is that this resolution does not match the quality of my Full HD monitor. Although the goggles have a higher resolution, they cover a wider field of view, which lowers the number of pixels per degree (PPD) because the pixels are spread over a larger area. In contrast, a monitor typically only covers 50-60 degrees of our field of view, resulting in a much higher PPD.
I don't see a compelling reason to switch from my monitor to goggles for work. Goggles do not offer higher resolution or richer colours compared to my monitor - at least for now.
Processing power
The Meta Quest 3 is twice as powerful as its predecessor and offers impressive performance as a standalone device. However, it still relies on a mobile chip and cannot cope with demanding tasks. For example, it struggles to edit photos or video due to the lack of dedicated apps, and even basic tasks such as editing in Canva via the browser were a poor experience, mainly due to the huge amount of lag.
Apple Vision Pro is equipped with M2 and R1 chips, making it much more powerful. However, the increased power comes with trade-offs, including a larger goggle size, higher cost, larger battery and more complex cooling. As microchip technology evolves, we can expect significant improvements in this area in the coming years.
System and integration - UI/UX
Of the problems listed, I believe that the UI/UX problem is the most important. It includes a number of frustrating inconveniences at the operating system level. These include minor glitches such as the keyboard not working properly, unintentional clicks when folding fingers while typing and a mouse cursor that does not always select the intended item. More serious problems include the lack of basic applications such as a calculator or notes, and an overall experience that feels more like an Android phone than a space computer. Additionally, problems such as unsynchronised items and a lagging remote display even with a strong Wi-Fi connection contribute to the overall discomfort.
There are a number of user interface issues that need to be resolved before goggles can be considered a serious working tool. While it is technically possible to achieve this experience by linking the goggles to your computer using applications such as Immersed, which allows you to create multiple screens and simulate the computer experience (as I tried, although it didn't work well for me), this approach somewhat undermines the idea of a spatial computer. Yes, it adds a 'spatial layer' to your computer experience, but it doesn't replace it entirely. Ideally, I would prefer a solution where the goggles, keyboard and mouse alone are enough for all my work needs.
While technically possible, numerous problems at the operating system level make this far from practical. The Met needs to reassess the purpose of this device and its integration within its ecosystem. The operating system should be more like a computer than a smartphone to enable effective working.
For example, when I connect my PC to Quest 3 using Remote Display, I have to switch the Bluetooth keyboard and mouse between Quest and PC every time I want to use them. Although there are buttons on both devices that allow you to switch between connections without re-pairing, this process undermines the advantage of having just one screen. Why should I have multiple floating windows on the Quest if I can't copy information from them and paste it into the PC? And why can't I move a window from the PC to my virtual space?
What needs to be improved and what the future holds
While it is tempting to simply say 'everything I mentioned above' when asked 'what needs to be improved', it is worth considering more aspects.
As well as Meta's focus on improving UI and UX, I'd like to see Codec Avatars finally make their way into standalone goggles.
Codec Avatars are one of the most promising technologies to emerge in the VR/AR space in the coming years, perhaps even decades. These avatars are highly realistic virtual representations of people, standing out from the more cartoonish avatars currently used by the Met. They achieve a level of hyper-realism that effectively overcomes the problem of the 'valley of the uncanny', a challenge that has plagued similar technologies in the past.
If Meta really refines the Codec Avatars for Meta Quest Pro 2 to be released in 2027, I believe we will see a gradual but growing interest in using VR/AR goggles throughout the day. Being able to have a friend or family member virtually appear in my room would offer a much better experience than video calls on a smartphone.
Another advantage of mixed reality goggles over traditional flat screens is the ability to interact with 3D elements in your environment. While you may have 2D screens floating around you, imagine being able to bring a 3D model into your space. This adds a new dimension to interacting with digital content.
It's certainly possible, and I believe that as mixed reality goggles become more popular, apps and web developers will start to explore creative ways to bring digital content from the screen into your 3D space. At the end of the day, I prefer to work on a computer rather than goggles. However, through my testing I have seen the potential that mixed reality goggles offer. Despite headlines suggesting that Meta is 'burning' billions on its Metaverse Reality Labs, I see growing competition and significant milestones being achieved in the industry. Mixed reality goggles are still relatively new, and this is the first time we have seen a goggle as complete at the operating system level as Apple Vision Pro.
I believe that by the end of the decade we will see a significant increase in the appeal of these goggles for applications beyond video games, such as work. I also expect Meta to start making profits from their VR/AR divisions in the 30s.
