Meta, the company formerly known as Facebook, has announced a significant artificial intelligence (AI) upgrade for its Ray-Ban Stories smart glasses, which first launched in 2021. The glasses now leverage new multimodal AI capabilities to see, hear and describe the world around users.
Key Highlights of the AI Features
The Ray-Ban Stories glasses use cameras, microphones and speakers built into the frames to enable hands-free photo and video capture, calls and more. With the latest upgrade, Meta is introducing the following key AI functionalities:
- Visual ID – Identifies objects seen through the glasses’ camera and provides descriptions aloud to users through the glasses’ speakers. It can recognize over 1,200 objects.
- Voice Commands – Allows hands-free control of the glasses through voice prompts like “take a photo” or “start recording a video.”
- Language Translation – Translates over 40 languages detected in text images or speech captured by the glasses’ cameras and microphones.
- Audio Captions – Automatically generates captions for audio like videos with speech. Captions are displayed on the glasses’ companion app.
The Road to Smarter Glasses
Meta first announced its partnership with Ray-Ban parent company EssilorLuxottica back in 2020 to develop smart glasses technology. The goal was to create stylish, lightweight eyewear that could seamlessly blend digital experiences with the real world.
The first-generation Ray-Ban Stories launched in 2021 focused mainly on photography and videography use cases. But Meta always envisioned infusing more cutting-edge technology over time.
| Year | Milestone |
|---|---|
| 2020 | Partnership formed between Meta and EssilorLuxottica |
| 2021 | Ray-Ban Stories smart glasses launch |
| 2022 | Additional style and prescription lens options introduced |
| 2023 | Major AI upgrade announced adding computer vision, voice commands and more |
This new AI upgrade represents the next phase of innovation, bringing more situational awareness and hands-free control to the glasses. It lays the foundation for even more future enhancements as the underlying technology evolves.
The AI in Action: Key Use Cases
The infusion of multimodal AI unlocks many new capabilities that aim to aid users throughout their day:
Visual Assistance
By identifying objects seen through the glasses’ camera, the Visual ID feature can provide helpful cues to the visually impaired or serve as a memory aid for forgetful users. It can read text on signs and menus, identify packaged foods, name plants and animals, and more.
Conversational Aid
The automatic audio captions create transcripts of conversations on the fly. This allows users to reference back details they may have missed or have a textual record to share.
The translation abilities also let users better communicate with those speaking other languages. This makes the glasses a powerful tool for traveling abroad.
Memory Capture
The enhanced voice commands allow for quicker, more seamless photo and video capture. Users can easily document moments without breaking the flow of an experience to manually operate a camera.
Over time, Meta aims to add capabilities to automatically curate media into collections around certain events or trips – reducing the need to sort through many similar photos afterwards.
What Critics Are Saying
Early feedback on the glasses’ AI upgrade has been generally positive, with some caveats around limitations:
- The Visual ID, while handy, only recognizes about 1,200 objects currently. More categories will need to be added over time to maximize usefulness.
- The translation feature requires a strong internet connection to function accurately, limiting its capabilities in areas with poor connectivity.
- The glasses still lack abilities compared to leading smartphone assistants like Siri or Google Assistant in areas like web search or deeper two-way dialogue.
- There are open questions around how expandable the core technology will be given the glasses’ lightweight form factor. More advanced AI may require heavier hardware than what the sleek design allows.
But overall critics found the current feature set compelling for lightweight, spontaneous use cases. And many believe Meta is taking steps toward enhanced AR glasses that may one day supersede smartphones as a primary mobile computer.
What’s Next?
For now, Meta is rolling out access to select Ray-Ban Stories users, with plans to open up availability more broadly over the coming months.
It also plans to continue enhancing the glasses’ capabilities over future software updates and next-generation hardware releases.
Areas of focus going forward likely include:
- Expanding the Visual ID catalog to recognize thousands more objects
- Adding more languages to translation abilities
- Enhancing voice interfaces for more flexible commands
- On-device processing for certain AI features to reduce reliance on connectivity
Ultimately, Meta is positioning these Ray-Ban Stories not as full-fledged AR glasses, but as a stepping stone toward that future goal. It provides an early glimpse of the merges of intelligent assistants and ever-present computing into stylish, wearable form factors.
And for Meta, each software upgrade and new model iteration brings them one step closer toward advanced augmented reality hardware – critical to fulfilling their vision of an embodied internet future.
So while imperfect as a first attempt, the infusion of multimodal AI into stylish, subtle packaging shows the potential for smart eyewear to become more than just a convenient camera. It may soon develop into an essential access point for information, connectivity and more as the line between technology and self continues blurring.
To err is human, but AI does it too. Whilst factual data is used in the production of these articles, the content is written entirely by AI. Double check any facts you intend to rely on with another source.
