The Information released a report on Apple’s AR/VR headset. Wayne Ma outlines the difficulties Apple faced during the development of its long-rumored AR/VR headset. The latest report corroborates Bloomberg’s 2020 report that Jony Ive had made a decision in 2019 to have development teams move away from products that work with external networks in favor of simpler, yet more self-contained devices.
In addition, the latest report also confirms that Jony Ive is still involved in the design and development of this product after leaving Apple. Apple CEO Tim Cook and then-Chief Design Officer Jonathan Ive tried to watch the VR demo on prototypes that simulated how the two approaches differed, according to two people familiar with the demo.
The headset used with the base station has excellent graphics, including realistic avatars, while the standalone version depicts its avatars more like cartoon characters. Apple VP Mike Rockwell, who runs the company’s AR/VR team, favors the web-enabled headset, arguing that Apple executives won’t accept a stand-alone version that’s too visually weak, according to two people familiar with the matter.
But he was wrong. Ive has been pushing for that standalone version since the early days of the project, according to a person familiar with the matter. Ultimately, Apple executives sided with Ive. Still, Rockwell promises he can make a great product. This also had a lasting impact on the multiple delays of the product. Allegedly, the product’s internal codename is N301.
His decision created a number of difficulties because “the development team has struggled to balance ‘range and performance while minimizing heat buildup so that people don’t get burned while wearing the device”. The source said the “high-quality mixed reality experience” promised by head of headset product Mike Rockwell to Apple executives as the main reason for the product’s multiple delays, as he was slow to deliver.
Apple executives expect their AR experience to far exceed those offered by rivals such as Facebook parent Meta Platforms in terms of graphics, body tracking and latency, especially the lag between a user’s movements and the real-time footage, three people familiar with the matter said. . IT Home understands that this is also one of the problems that most current VR products cannot overcome.
There was also disagreement among the product teams on the targeting of customers. Some say Jony Ive made the wrong decision to go from a high-end web-enabled headset to a standalone device.
Some of them put the blame on Ive, who they say has fundamentally changed the purpose of the product, from a product used at the desk by creatives and professionals to a portable device that the mass consumer can afford. These people believe that Apple should first develop a product for professionals, encouraging them to open content channels for Apple’s mixed reality ecosystem, and then release a product suitable for them, consumers.
It is said that this Apple headset will feature more realistic avatars, so 14 cameras will be required to capture accurate facial expressions.vAnother challenge that has greatly influenced the development of Apple’s headset technology is its 14 cameras, which allow it to capture everything possible, whether it’s images of the outside world or the user’s facial expressions and body poses.
Apple has to use the Bora image signal processor to process a lot of images, but Apple engineers had trouble getting Bora to work with the headset’s main processor (codenamed Staten). The back-and-forth communication between the two chips adds latency, which can make the person wearing the headset nauseous.
That means Apple has to overcome this problem in another way, the streaming codec, but it’s far from the end. The bigger problem facing the project is a lack of discipline. Before 2019, its culture was freewheeling, operating almost like a startup inside Apple, four people familiar with the team said. The employees brainstormed and tried scenarios that might never have appeared.
As a result, Apple hired veteran Kim Vorrath to manage and lead the advancement of the project and later assigned hardware chief Dan Riccio to oversee the development of mixed reality products.
Vorrath did bring more structuring to the Apple team, and she asked individual teams to come up with defined capabilities for the headset software to motivate them and increase their accountability. After she joined, engineers were indoctrinated by her with a new concept, one she often uses in software engineering, the “six-week sprint,” two of the people said.
There’s also a small episode where Jony Ive is still working on the project even after he left Apple. For example, some development team members had to go to his home to get “change approval”.
A person familiar with the matter said that Ive will continue to provide consulting work for Apple after his departure, including the headset product. The source added that he was often called on to help his former team push their preferences in areas such as batteries, camera placement and ergonomics, rather than engineers’ preferences. Even after Ive left Apple, some employees on the headphone project still had to make the long journey from Cupertino to Ive’s home in San Francisco to get his approval for the changes, the two said.
Another tidbit Ive has been tweaking the headset design over the years, and his most recent preference is for the headset to have the battery tethered and worn next to the headset, rather than having it integrated into the headband.
Finally, another challenge is cost. That’s one of the main reasons Apple assigned executive Dan Riccio to the project. Reports from Bloomberg and The Information suggest that Apple currently has a price target of between $2,000 and $3,000.