Apple’s long-awaited mixed reality headset is the company’s ambitious attempt to create a 3D version of the iPhone operating system with eye and hand tracking systems that could make the technology stand out from competitors’ products. The roughly $3,000 device due later this year, likely called Reality Pro, will show a new approach to virtual meetings and immersive video to shake up the Meta-dominated VR industry.
Apple enters an uncertain market with a premium product. The company’s technology development team of more than 1,000 people has spent more than seven years on the project, and the company expects it to become a new source of revenue, especially given that sales growth will stall this year.
But virtual reality has proven to be a challenge for the greatest titans of technology. While some predict the industry will top $100 billion by the end of the decade, headsets are still considered niche products—and Meta has lost billions in its efforts.
Meta headset with hand controllers. Source: Bloomberg
Apple’s goal is to introduce something new. According to people familiar with the product, eye and hand tracking capabilities will be the main selling point of the device. Its main features will include advanced video conferencing and FaceTime-based meeting rooms. The headset will also be able to display immersive video content, serve as an external display for a connected Mac, and duplicate many of the features of an iPhone and iPad.
The headset will receive several external cameras that can analyze the movement of the user’s hands and sensors in the body of the eye-reading gadget. This will allow a person to control the device by looking at an item on the screen—be it a button, a program icon, or an item in a list—to select it. To activate the task, the user must simply bring the thumb and forefinger together without the need to hold something. This approach is different from other headsets that typically rely on a handheld controller.
Like the latest Meta headset, Apple’s device will use both virtual and augmented reality. With virtual reality, users see the image and content while wearing glasses. In addition, augmented reality superimposes digital content on the image of the real world.
The headset received two ultra-high-resolution displays, developed in conjunction with Sony, to work with virtual reality and a set of external cameras to provide pass-through AR mode. This means that users will see the real world through the cameras on the headset. Apple will offer users with eyeglasses special lenses that are located in the case itself.
The device will feature the Digital Crown, an Apple Watch-like wheel that lets you switch between VR and AR. Apple expects this to make the product stand out from others.
The FaceTime software will realistically display the user’s face and entire body in virtual reality. These avatars will allow two people – each with an Apple headset – to communicate immersively. The technology differs from the virtual meeting rooms on the Meta headset, which creates a more cartoon-like user avatar.
Due to the massive processing power required for this feature, the headset will only support realistic avatars during one-on-one video chats. This will still allow FaceTime sessions with multiple people, but additional users will appear as an icon or Memoji – Apple’s customized emoji.
Animoji when making an iPhone FaceTime call. Photographer: David Paul Morris/Bloomberg
Apple plans to introduce the device this spring, although the schedule is still subject to change. This will allow the company to discuss the product at the annual Software Developers Conference in June and then release it later this year. Some features may still be canceled or changed, Bloomberg sources say.
The company may start early production of the device in China as early as February, and is considering launching the product in the US only to begin with. The price is also expected to limit the product’s appeal, and Apple is already working on a cheaper version – to be released in late 2024 or early 2025 – that will cost closer to $1,500.
Apple expects to sell about a million units of its new headset in its first year. That compares to more than 200 million units of the iPhone, the Cupertino-based company’s biggest earner. However, Apple does not plan to immediately turn a profit on the initial version – even at a high price – indicating that the company has long-term plans for the platform.
The main function of the new device will be watching videos with an immersive effect. Apple has discussed developing VR content for the platform with a dozen media partners, including Walt Disney and Dolby Laboratories. The tech giant itself is working on updating its own Apple TV+ content to work with the headset. In 2020, Apple acquired streaming company NextVR to create sports content in VR.
Apple plans the headset to have a dedicated video viewing feature that will make viewers feel like they’re watching a movie on a giant screen in a different environment, like the desert or outer space. However, the headset’s speakers may not be able to handle it, so users will need to wear AirPods to get the full surround sound experience.
The device will also get productivity features, including the ability to serve as an external monitor for the Mac. With this feature, users will be able to see the computer display in virtual reality, but still control it with a trackpad or mouse and physical keyboard.
The headset’s operating system, called xrOS, will have many of the same features as the iPhone and iPad, but in a 3D environment. These include the Safari web browser, Photos, Mail, Messages, and the calendar app, as well as apps for company services such as the App Store for installing third-party software, Apple TV+, music, and podcasts. The company is also working on health tracking features.
The experience should be familiar to Apple users. When they put on the headset, the main interface will be almost identical to that of the iPhone and iPad, showing a main screen with a grid of icons that can be rearranged. Users will be able to pin widgets such as weather, calendar appointments, email, and stock market numbers to their app icons.
When users need to enter text, they can use the Siri voice assistant or the iPhone, Mac, or iPad keyboard. However, unlike the Apple Watch, you don’t need an iPhone to work. The company is developing technology that will allow users to type in the air with their hands, but such a feature is unlikely to be ready for the first launch.
Games are expected to become a popular offering from third-party developers. In 2017, the company released ARKit and other tools to help prepare the augmented reality experience on the iPhone. This provided a basis for programmers to create programs, games and services for the headset.
Apple’s device will include a variation of the M2 chip found in recent Macs, as well as a dedicated processor for graphics and mixed reality. This second chip will be called Reality Processor, according to trademark applications filed by the tech giant.
But building sufficiently powerful processors caused another problem: the device overheated when placed on the user’s face. To solve this problem, Apple decided to make an external battery that is in the user’s pocket and connected with a cable. Another change is the inclusion of a cooling fan, like on high-end Macs.
The headset can operate for about two hours on a single battery, similar to competitors’ products. However, the battery is large: about the size of two iPhone 14 Pro Max stacked on top of each other, or about 15.2 cm high and 1.27 thick. However, some internal prototypes for software development do have a built-in battery and charge via USB-C.
Meta Quest headset. Photographer: Nic Coury/Bloomberg
Meta, in contrast, places its batteries on the back of the headset in a way that helps balance the device on a person’s head. Apple’s approach may be less user-friendly, especially if they’re watching a movie in a headset. Some testers complained that the product was bulky.
The relatively short battery life – about 20 hours less than Apple’s latest MacBook Pro – could create other problems. If users want to watch movies or play games for hours, they will have to purchase multiple batteries and change them frequently.
Apple recognized these challenges within the company and tried to set realistic expectations for the product. One advantage of the device, according to the company, is that it can encourage customers to visit Apple retail stores to try out the product and purchase another device, such as an iPad or AirPods.
The initial headset will be made of aluminum and glass – and will resemble Apple’s $550 AirPods Max. The product will receive a curved screen on the front, speakers on the sides and a special headband that helps place the device on the head. This will set the product apart from the mostly plastic design of competitors, who usually “tie” the device to the user with multiple straps.
Eye and hand tracking systems can be the most original elements of the headset. Apple likes to add an innovative interface that sets its products apart from the competition: the iPod had a wheel, the iPhone and iPad had a multi-touch approach, and the Apple Watch had the Digital Crown. Now the company is hoping that with the headset’s sci-fi interface, its latest product will also take the lead.
Source: Bloomberg
