Lux Research - The Hardware and Materials Innovations ...

1

The Hardware and Materials Innovations Enabling Consumer Augmented Reality

Vladimir Roznyatovskiy

Senior Analyst

Lead Analyst:

Image: Microsoft

E X E C U T I V E S U M M A R Y

Client confidential. Not for redistribution.Client confidential. Not for redistribution.

Executive Summary

2

Consumer mass-market augmented reality (AR) products and applications could become a fourth wave of modern technological progress, following the PC and smartphone in transformative impact on nearly all aspects of life and business.

However, making widespread consumer AR a reality will require significant

Maturity of AR components for mass-market usesText gives current readiness, while color shows full innovation potential

Optical Components Light Engine Sensors and

CamerasWaveguides Films MicroLED Lasers

Comfort Medium High Medium High Low

Immersion Medium Low Medium High High

Cost* High Medium High Medium Low

* Inverted color scheme: The lower the cost, the higher the color score

innovations in optical components, light engines, and cameras and sensors to reduce size and weight while improving performance. These advances will create significant opportunities for materials and hardware developers. It will be five years to seven years before these efforts translate into viable mass-market devices, but once they do, the opportunities for consumer insights and engagement will be tremendous and a significant source of competitive differentiation.


Global technology trend: The fourth wave of computing?

3

Mainframes Smartphones

Computing power for many but immobile and with no contextual awareness

Personal computers Mixed reality

Ubiquitous computing at the edge that blends the physical and digital worlds

Computing power for all with on-the-go access and some contextual awareness

Computing power made possible for the first time by tethered terminals

1970s 1990s 2000s Today

Demand for improved productivity and flexibility drove multiple innovative iterations in computing technologies and data delivery methods. What’s the next wave? The combination of mobility and spatial visualization, along with elements of immersion, could enable emerging augmented reality (AR) or mixed

reality to have as great an impact as previous waves.


Impetus for change: AR use cases

4

AR has the greater potential return on investment (ROI) in enterprise uses – the tech lacks sufficiently comfortable AR hardware today to serve entertainment or other consumer applications. Some key commercial use cases include:

Reduce production time

Increase skills and efficiency

Train and onboard using interactive scenarios and environments to reflect real-life situations, improving retention and effectiveness

Win new customers

Uncover actionable insights

Make pertinent data available in context, enabling better, faster, more informed decisions

Delight customers and build brand loyalty with immersive 3D experiences that create memorable, high-impact impressions

Empower the workforce to solve problems in real time, with hands-free access to essential information and remote expertise

Workforce transformation

Simulation training Data in context with AI

Customer experiences


AR component maturity is determined by level of comfort, immersion, and price

5

AR glasses’ readiness for the mass market is limited by several factors, including comfort (wearable, vestibular, visual, and social), depth and quality of experiences (immersion), and technical complexity affecting individual components and overall cost. The physical appearance should approach that of regular corrective glasses or sunglasses, and the weight should be less than 80 g.

Comfort

Vergence-accommodation conflict mitigation

Motion to photon latency (<10ms)

Untethered (no cables)

Size and weight

Ghost, pupil swim

Thermal management

Brightness and contrast

Vision impairment correction

Immersion

Field of view (FoV)

Gesture sensing

Angular resolution (>45 pixels/degree)

Arm’s length interactions

Active dimming, occlusion

Foveated and peripheral displays

World-locked spatial audio

Fast and accurate eye tracking

Cost

This report expands on the major components that individually and together deliver AR experiences and details the current limitations and major trends in AR components.

Development expenses

Manufacturing cost


Balancing requirements for consumer AR

6

Vuzix smart glasses without world tracking

Enterprise AR headwear

• Provides 6DoF tracking

• Heavy and bulky

• Cost is over $2,000

Consumer AR headwear

• Limited in performance and immersion

• Approaches weight of optical glasses

• Cost comparable to a smartphone

HoloLens 2 with world tracking

Cost andwearablecomfort

Visualcomfort andimmersion

The best-in-class AR headset today, the Microsoft HoloLens 2, weighs more than 500 g; the second generation of Magic Leap announced to come out in Q4 2021 is planned to weigh 250 g. Both focus on enterprise markets. Hands-free consumer AR today is largely confined to smart glasses (<100 g) for daylong use and offers limited performance. Combining today’s enterprise performance and consumer hardware comfort requires significant innovations in materials and manufacturing before ubiquitous AR glasses enter the mass market.

https://www.roadtovr.com/magic-leap-2-release-date-specs/


What will it take for AR to go mainstream?

7

Achieving the needed combination of enterprise performance requirements and consumer headwear comfort means that significant hardware innovations will have to occur prior to widespread AR glasses adoption.

Realizing mass-market AR headsets will require significant efforts to miniaturize components and simultaneously improve functions. Innovations in light engine, optical, and sensory components offer the greatest materials opportunities.

This report discusses how individual major components and their functions translate into AR experiences and details the current challenges in this space.


Among AR hardware components, optics, sensors, and light engines offer the greatest potential for innovations

8

Achieving both comfort and immersion levels at a reasonable cost requires miniaturization of all the components with simultaneous performance improvements. Optics, light engines, sensors, and batteries take up the largest amount of physical space and, therefore, offer the greatest potential for innovations. Lux has published extensive research on batteries elsewhere, but suffice to say here that ongoing efforts to improve lifetime and weight for other consumer electronic devices will benefit AR glasses as well. This report focused on ongoing efforts and key players in optics, light engines, and sensors and cameras. Along with analysis of the functions that the components perform, the purpose of this report is also to evaluate maturity and the potential for innovations for each type of component.

Optics

• Waveguides

• Holographic optical elements (HOE)

• Traditional light shaping mirrors

• Metamaterials

Light engine

• MicroOLEDs

• MicroLEDs

• Laser based displays

Sensors and cameras

• RGB and NIR cameras

• TOF range sensors

• Other tracking sensors (eye and UWB sensors)

https://members.luxresearchinc.com/research/technology/Li_ion_batteries

https://members.luxresearchinc.com/research/report/35392


AR optics: What you need to know

9

COMMERCIALIZATION: Present to 2025

WHAT IT IS KEY POINTS AND MATURITY LUX TAKE

Components that display a virtual image focusing either on or near the user’s eye(s). Typically, waveguides or holographic mirrors serve to overlay and blend digital information with physical world appearance. Currently, fully immersive AR headsets are based on waveguides, with limited field of view, while holographic elements are being developed for smart glasses and automotive applications.

Waveguides are available today but require improvements in weight and materials performance to increase FoV, as well as other quality and immersion characteristics.

Holographic optical elements (HOEs) serve well for smart glasses without real-world overlay and can be applied on curved prescription glasses. HOEs will gain adoption in the next few years as a stopgap measure until waveguides fully mature.

HOEs are best-suited for smart glasses applications without world tracking in the near term, while waveguides will take the spotlight once the performance-quality balance reaches its full potential and requirements for consumer AR. Materials and processing innovations will contribute the most to this transformation. Developments from chemical companies like DuPont and Covestro, as well as internal efforts by Apple, Facebook, and Google, will offer opportunities for partnerships.


Y-axis: Summary of trends in patents, papers, funding, & more.

“Innova

tion

Inte

rest”


“Innova

tion

Inte

rest”

10

Lux Tech Signal: AR optics

AR components have seen increasing innovation interest over the years. Optical elements require individual development (unlike some of the display components common in other applications) and have drawn substantial effort compared to the overall AR technology progress.

AR/VROptics for AR/VR


Strengths

Technology evaluation: AR optics

11

Weaknesses

• Can be manufactured with high precision and volume using established processes

• Sufficient eye-box size

• Currently insufficient field of view (FOV)

• Prescription lens integration challenges

• Currently heavy

• Made with two sublayers for red-green and green-blue colors separately

Strengths

Weaknesses

HOE films (mirrors)Patterned waveguides

• Lightweight

• Thin

• Adaptable to curved surfaces

• Easily integrate with prescription lenses

• Very sensitive to facial dimensions/foveated rendering

• Applicable to smart glasses without world tracking

• Sensitive to environmental fluctuations


Light engines: What you need to know

12

COMMERCIALIZATION: Present to 2027

WHAT IT IS KEY POINTS AND MATURITY LUX TAKE

MicroOLEDs are being used for VR and passthrough AR as a stopgap technology. They can form pixelated microdisplays and are best-suited for smart glasses but today only offer single-color, not RGB, versions.

Laser sources provide discrete, pure, and coherent illumination that is being rasterized by means of MEMS to generate virtual images. They can deliver immersive experiences but require further miniaturization for consumer applications.

Key elements in creating an augmented reality experience are light sources and light steering devices that convert image data into a light object that is later focused by optics to be perceived as a virtual image. Current technologies rely on laser beam scanning (LBS) and microOLED or liquid crystal on silicon (LCoS) architectures – though the latter two have a limited path to miniaturization.

In the near term, green light microLED will gain adoption in smart glasses in parallel with LBS. RGB smart glasses will see adoption only in three to four years, when RGB microLED fabrication yield improves. At that point, the microOLED market share will decrease.

LBS will enable fully immersive AR glasses when both lasers and optics, including waveguides, resolve size and chromatic bottlenecks, which will not be until 2026 at the earliest.


Light engines: Six major display architectures

A variety of displays have been developed for AR uses, including conventional liquid crystal (LCD) and organic LED (OLED) as well as LCoS and digital light processing (DLP).

However, only microLED and LBS carry the potential for further improvement to offer sufficient brightness within a miniature and power-efficient form factor. Microdisplays based on OLED are seeing interim adoption in VR and video-passthrough AR headsets.

13

DLP micro-display

OLED

RGBcombiner

MEMSmirror

RGBLasers

Siliconsubstrate

Mount

LCD MicroLEDLCoS

OLED Laser beam scanningDLP



“Innova

tion

Inte

rest”


“Innova

tion

Inte

rest”

14

Lux Tech Signal: Light engines

Augmented reality and light engines received close to $12 billion in investments over the past decade. The funding came predominantly from U.S. venture capital for American-based developers.

AR/VRLight engines for AR/VR


Strengths

Technology evaluation: Light engines

15

Weaknesses

Strengths

Weaknesses

Laser beam scanning (LBS)MicroLED

• High brightness

• High resolution built into the microLED-based display

• Fast response time

• Small size and weight

• Costly manufacturing with color and performance uniformity challenges

• Thermal management is unresolved

• Full-color RGB displays cannot easily be made

• Very bright illumination

• Coherent illumination offering the purest colors

• Relatively high power efficiency

• Fast response time

• Requires three color laser sources and a combiner

• Requires a complex system for laser beam steering mirrors (MEMS) and optics to shape the wavefront


Cameras and sensors: What you need to know

16

COMMERCIALIZATION: 2022 to 2027

WHAT IT IS KEY POINTS and MATURITY LUX TAKE

Infrared (IR) sensing is currently limited to capturing eye movement with special-purpose devices. Extending the capabilities of regular cameras into the IR spectrum will allow for better object detection.

Metalenses allow dramatic shrinkage of optical components and overall reduce the form factor of cameras and sensors; however, they require custom design for each use case –currently not available for scale.

Devices measuring images and distances to surrounding objects are needed for combining the virtual and real worlds. The applications range from tracking hand gestures to 3D mapping to object recognition and many more. Currently, far-range measurements are based on multiple cameras, while short-distance sensing employs time-of-flight (ToF) systems.

Cameras and sensors will see further improvement with decreasing size and cost of manufacturing.

Quantum dots will add IR sensitivity and provide additional insight from the same capture data, though fabrications methods will require one year to two years before entering consumer products.

Metalenses are highly promising and extremely difficult to design, with mass-market devices still three to five years away.


Cameras and sensors: Short- and long-term opportunities

17

Consumer AR will require a set of cameras and range sensors to differentiate between materials, objects, and dynamic weather conditions. Better texture detection, including near-range hand tracking and distant object semantic understanding, makes for a better immersive experience. Metamaterials, IR sensing, and wafer-level optics processes promise to improve the cost and form factor.

The current process relies on using III-V materials flip-chip bonded on the camera CMOS. These composites harvest IR energy through a photovoltaic effect. Emerging options replace III-V materials with quantum directly on the CMOS.

CCD or CMOS

InGaAs InSb or HgCdTe

Layered metalensesimage sensor

Incumbent lenses set

Moldable/wafer-level optics MetamaterialsIR sensing

Wafer-level optics are optimized for basic microlenses. Complex elements requiring improvements in fabrication are diffractive optical elements (i.e., structured light), ultraflat cluster cameras (i.e., light-field cameras), and 3D (SLA) printing of optical elements, especially nonsymmetrical (currently molding and precision machining of glass/plastic).

Optical metamaterials use micro/ nanoscale patterning to manipulate visible light with greater control than conventional optics, with lower weight, smaller size, better imaging, and greater design freedom.

https://members.luxresearchinc.com/research/report/39411



“Innova

tion

Inte

rest”


“Innova

tion

Inte

rest”

18

Lux Tech Signal: Cameras and sensors

AR/VRCameras and sensors for AR/VR

Camera and sensor component development requires capex-heavy efforts typically made by large companies. Nevertheless, more than $50 million from materials- and hardware-focused VCs in the U.S. and Europe was spent on startups with increasing effort within the past several years.


Strengths

Technology evaluation: Cameras and sensors

19

Weaknesses

Strengths

Weaknesses

MetalensesIR sensing

• Provides additional textural and spatial information using the infrared spectrum

• Can be incorporated as part of an RGB camera sensor or a dedicated IR sensor

• Incumbent IR sensing materials/technologies are bulky and expensive

• Quantum dots are a promising substitute but are not mature enough for the applications

• Very small form factor

• Rely on established semiconductor fabrication processes

• Can be designed to conform to specific performance with smaller chromatic aberrations compared to traditional optics

• Early-stage technology without mass-market product

• Metalens design requires advanced and specialized skills with sparsely available talent


Summary: The maturity of AR components varies, with many having significant potential future growth

20

All basic components for functional and immersive AR headwear are available today. Several large corporations, such as Microsoft and Lenovo, are tuning these products for enterprise applications.

However, the transition to consumer AR glasses requires significant

Maturity of AR components for mass market usesText gives current readiness, while color shows full innovation potential

Optical Components Light Engine Sensors and

CamerasWaveguides Films MicroLED Lasers

Comfort Medium High Medium High Low

Immersion Medium Low Medium High High

Cost* High Medium High Medium Low

* Inverted color scheme: The lower the cost, the higher the color score

innovations in optical components, light engines, and sensors in order to achieve the desired combination of immersion and comfort prior to entering the mass market.

The progress will take several years, depending on the maturity of each of the components and potential for improvements. The market evolution will be staged, with the release of limited-function products like AR smart glasses first and with fully immersive and comfortable AR glasses to follow.


Outlook

21

1Video pass-through AR will gain traction with consumers in the next two to three years. VR products like Varjo, Lynx, and anticipated Apple headsets with real-world video displayed on microOLED displays will serve as a stopgap, filling the consumer need for immersive and reasonably comfortable daily experiences. The applications will be limited to indoors and focus on entertainment.

2Smart glasses adoption will begin in the next two to five years. Progress in microLED and AR optics will lead to volume sales of smart glasses like Vuzix and Focals. Limited functions like notifications, navigations, and productivity assistance will still depend on smartphone connectivity but will further accustom consumers to unobstructed information on the go. This period of AR expanding to outdoors will initialize location-based AR experiences.

3Fully immersive and slick AR glasses capable of blending virtual and real worlds will likely come in five to seven years. Optimized AR components will enable world tracking (6DoF) and completely immersive day-long experiences. This period will present substantial opportunities for materials and hardware developers, while the world afterward will present unlimited virtual opportunities for development.

Innovate Smarter & Grow Faster With Lux

Lux Research, Inc.

YouTube:

Lux Research

@LuxResearch

Blog:

Lux Blog

Free Webinars:

Lux Webinars

@LuxResearch

Contact us:

www.luxresearchinc.com| [email protected]

https://www.linkedin.com/company/lux-research/

https://www.youtube.com/channel/UCrkChLQkJGvE2ND8W41CKmQ

https://twitter.com/LuxResearch

https://www.luxresearchinc.com/blog

https://www.luxresearchinc.com/webinars

https://www.facebook.com/LuxResearch/

Lux Research - The Hardware and Materials Innovations ...

Documents

Transcript of Lux Research - The Hardware and Materials Innovations ...