It seems like all the big companies are taking an interest in Hybrid Audio Systems Technology. In our last article, Apple Creates Another New Patent: Air & Bone-Conduction Headphones we recently talked at length about Apple and their journey from computing giant to the masters of digital music.
Apple is always on the lookout for or pioneering the next big innovation in mobile or digital music technology. Or more specifically, the ways we listen to music and the devices we utilize, such as headphones.
We’ve discussed how in 2016 then again in 2020 Apple patented new concepts for Bone Conduction headsets. In the years since the original patent was filed in 2016, no plans had come to fruition. We’ve speculated on why this might be, coming to the conclusion that the market and Apple themselves may not be ready to dip their toe into the Bone Conduction arena just yet.
Bone Conduction Headphone Market is Growing
The Bone Conduction market is growing, companies like AfterShokz have their target audience, but it’s still considered a niche product. Bone Conduction headphones are waiting for their moment to break out, we’re sure it’s coming at some point. But it’s going to take a large company like Apple, or one of their competitors to fire the first shot. So far everyone has been unwilling.
However, there seems to be a new contender on the horizon, and Bone Conduction technology among a host of others with Hybrid Audio Systems Technology may get their next big break from an unlikely source; Facebook.
Facebook Technologies, LLC. and Oculus (formerly known as Oculus VR, LLC) have filed a multitude of patents for Hybrid Audio Systems within Augmented Reality Devices (AR), Virtual Reality Devices (VR), Smartglasses, and multiple wearable devices. While Apple filed its own first patent for Bone Conduction technology back in 2016. Facebook on the other hand has been filing their patents constantly since then and much more recently with just a partial list below.
Facebook Technologies Hybrid Audio Systems Wearable Devices AR/VR, & Glasses Partial Patents List
- U.S. Patent US10070224B1 2018-09-04 Granted Facebook Technologies – Crosstalk cancellation for bone conduction transducers
- U.S. Patent US10419843B1 2019-09-17 Granted Facebook Technologies – Bone conduction transducer array for providing audio
- U.S. Patent US10231053B1 2019-03-12 Granted Facebook Technologies – Bone-conduction headset with crosstalk cancelation function
- U.S. Patent US10231046B1 2019-03-12 Granted Facebook Technologies – Cartilage conduction audio system for eyewear devices
- U.S. Patent US10261542B1 2019-04-16 Granted Facebook Technologies – Distributed augmented reality system
- U.S. Patent US10531173B2 2020-01-07 Granted Facebook Technologies – Apparatus, system, and method for tragus conduction hearable device
- U.S. Patent US10555106B1 2020-02-04 Granted Facebook Technologies – Gaze-directed audio enhancement
- U.S. Patent US10684690B2 2020-06-16 Granted Facebook Technologies – Methods, devices, and systems for creating localized haptic stimulations on a user
- U.S. Patent US20200111259A1 2020-04-09 Pending Facebook Technologies – Apparatus, systems, and methods for display devices including local dimming
- U.S. Patent US20190342647A1 2020-08-25 Granted Facebook Technologies – A hybrid audio system for eyewear devices
- U.S. Patent US10812929B1 2020-10-20 Granted Facebook Technologies – Inferring pinnae information via beamforming to produce individualized spatial audio
From the Facebook Technologies United States Patent US10684690B2 for Methods, Devices, And Systems For Creating Localized Haptic Stimulations on a user we get the following statement that shows the design concept to incorporate audio output and input:
“Artificial reality systems may also include one or more input and/or output audio transducers. In the examples shown in FIGS. 13 and 15, output audio transducers 1308(A), 1308(B), 1306(A), and 1506(B) may include voice coil speakers, ribbon speakers, electrostatic speakers, piezoelectric speakers, bone conduction transducers, cartilage conduction transducers, and/or any other suitable type or form of audio transducer. Similarly, input audio transducers 1310 may include condenser microphones, dynamic microphones, ribbon microphones, and/or any other type or form of input transducer. In some embodiments, a single transducer may be used for both audio input and audio output.”
Facebook Technologies United States Patent US10684690B2 for Methods, Devices, And Systems For Creating Localized Haptic Stimulations
Oculus Virtual Reality Headsets
Founded in 2012, Oculus VR produces virtual reality headsets, including the Oculus Rift and Oculus Quest lines.
So What Do We Know?
Well for starters these patents aren’t for a set of typical headphones, but they will work like one. They Are in fact for sets of Augmented Reality Glasses that are designed for use in gaming, virtual reality, and any other fun ideas Facebook comes up with.
Should they ever come to market, the glasses will make use of the following technologies for the benefits it brings to hearing sound.
Output Audio Transducers
- Voice Coil Speakers
- Ribbon Speakers
- Electrostatic Speakers
- Piezoelectric Speakers
- Acoustic Sensor
- Air Conduction Transducer
- Bone Conduction Transducers
- Cartilage Conduction Transducers
- Beamforming
- Gaze-directed Audio Enhancement
- Any other Suitable Type or Form of Audio Transducer
Input Audio Transducers
- Condenser Microphones
- Dynamic Microphones
- Ribbon Microphones
- Any other Type or Form of Input Transducer
In combination or by themselves, all of these offer a unique way to hear the sound, allowing the user almost to feel the sound like the different types and forms of audio signal permeate through our skulls and into our inner ear. Like Bone Conduction, it’s an immersive feeling that one needs to experience to appreciate it truly.
Facebook Technologies Testing Hybrid Sound Technologies
We can see why Facebook is testing all of these technologies to get the consumer the most prolific multiple ranges of sound experience possible by adding it to their devices. Augmented reality and virtual reality are both designed to be immersive experiences and using Multiple Hybrid Audio instead of traditional sound design is a fantastic way to achieve this goal.
Facebook has chosen Air Conduction, Bone Conduction, Cartilage Conduction, Beamforming, and Gaze-directed Audio Enhancement technologies for this very reason. But also, because it allows outside sound to still be heard.
Many of these audio technologies, like Bone or Cartilage Conduction, are not noise-canceling; in fact, they are quite the opposite. Part of their appeal is they allow sound from the outside world to still be heard. This allows those using them to retain some sense of what’s going on around them, but due to the sound’s deep and penetrating nature, they are still able to focus on it and enjoy the music, audio, augmented reality without becoming distracted.
It does however mean the user can keep their wits about them, which is exactly what someone experiencing virtual or augmented reality needs to do. Becoming too immersed could be dangerous to ourselves and others.
Facebook is obviously serious about these designs and has completed multiple redesigns of these patents. Like Apple, Facebook is aware of technologies and are hard at work thinking up ways to use them and ultimately bring them to market. These AR glasses may never see the light of day, or they could be unveiled and released in the very near future.
Tell Me More About Facebook’s Hybrid Audio System For Eyewear Device
The images on the patent show what looks like a normal pair of glasses from a front-facing angle. But a side view gives us a much better look at how they will work, especially how they will deliver sound. The glasses themselves don’t actually resemble a pair of Bone Conduction headphones but in a way the reverse. Normally Bone Conduction headphones are attached to a headband that snuggly fits behind the head and cups over the ears, and the vibrating speaker neatly slots into the temple or soft bit of flesh we all have in front of our ears.
The glasses work similarly, but they connect from the front rather than the back because they are a pair of glasses. But the principle remains the same. Most glasses fit on our heads by having their folding parts slide over our ears. Facebook’s devices do this too, but, for added sound support, they fold around our ears depending on the design.
The patent shows four different types of audio input, an Acoustic Sensor in the ear canal, an Air Conduction Transducer directly below the ear canal, Bone Conduction Transducers, and Cartilage Conduction Transducers located at what they call the back temples of the glasses and the temple tips. Facebook has labeled its technology “Hybrid Audio System For Eyewear Devices” because it can be applied to many styles and devices that attach to the cranium. This Hybrid Audio System could be applied to attachments similar to Bone Conduction Headphones as well.
A few inches away from the traditional speaker, within our skull, sits our cochlear. The organ is responsible for converting soundwaves into brain waves. It is basically the brain of the ear and our sense of hearing in general. It regulates and translates sound into something we can understand. We normally hear sound waves through our ears, which is then picked up and transmitted to the cochlear by our eardrum. But Bone Conduction works differently, this works by letting the vibrations travel through our body until they reach our skull and the cochlear within. So, when we say Bone Conduction lets you feel’ the sound, you know we’re not joking. You literally are.
This is what these Air, Bone, and Cartilage Conduction technology takes advantage of. It’s been used in the military and in hearing aid technology for decades, but in recent years the audio/music technology industry has seen its value and attempted to monetize it by selling the technology to the masses. But in a way, Transducer Conduction technology still remains one of sound’s best-kept secrets.
Gaze-directed Audio Enhancement
The Gaze-directed Audio Enhancement design is coordinated with behavior that will play an important role in future technology for these Smart Glasses, Augmented Reality Devices (AR), Virtual Reality Devices (VR), and hearing aids.
“Embodiments relate to a headset that filters sounds according to a direction of a gaze of a user wearing the headset. The user wears the headset including an eye tracking unit and one or more microphones. The eye tracking unit tracks an orientation of an eye of the user to determine the direction of the gaze of the user. The direction of the gaze may be different from a facing direction of the headset. According to the determined direction of the gaze of the user, input sound signals generated by the microphones can be beamformed to amplify or emphasize sound originating from the direction of the gaze.”
U.S. Patent US10555106B1 Facebook Technologies – Gaze-directed audio enhancement
Although this is a Patent it is likely that this would eventually take on a similar design as the Hybrid Audio System For Eyewear Device as it states in the speaker description section with a quoted excerpt below.
“In some embodiments, the speaker 180 is based on bone conduction or cartilage conduction that keeps the ear canal open.”
U.S. Patent US10555106B1 Facebook Technologies – Gaze-directed audio enhancement
Inferring Pinnae Information Via Beamforming To Produce Individualized Spatial Audio
Spatial audio techniques to improve clarity and depth are used in consumer audio applications in smartphones and headphones. Spatial audio lets you position sounds anywhere in 3D space, which is essential to the sound’s design and clarity from a set of headphones, hearing aid, or now Smartglasses.
Below an excerpt from the Facebook Technologies patent for Inferring Pinnae Information Via Beamforming To Produce Individualized Spatial Audio discusses a bone conduction transducer, cartilage conduction transducer, or multiple speakers integrated into the frame of the Smartglasses or headset.
“The transducer array presents sound to the user. The transducer array includes a plurality of transducers. A transducer may be a speaker 160 or a tissue transducer 170 (e.g., a bone conduction transducer or a cartilage conduction transducer). The speakers 160 may be enclosed in the frame 110. In some embodiments, the headset 100 includes a speaker array comprising multiple speakers integrated into the frame 110 to improve directionality of presented audio content. In some embodiments, the speakers 160 may each be placed within an ear canal of the user. The speakers 160 may be positioned at other locations of the headset 100. The tissue transducer 170 couples to the head of the user and directly vibrates tissue (e.g., bone or cartilage) of the user to generate sound. The number and/or locations of transducers may be different from what is shown in FIG. 1A.”
U.S. Patent US10812929B1 Facebook Technologies – Inferring pinnae information via beamforming to produce individualized spatial a
How This May Benefit The Headphone Industry
In our previous article regarding Apple Creates Another New Patent: Air & Bone-Conduction Headphones, as we discussed that the marketplace may need a catalyst, “a match to light the bonfire” was the term we used. Facebook unveiling their Hybrid Audio Systems For Eyewear Devices glasses, Gaze-directed Audio Enhancement, could be exactly that.
Once the items are unveiled and released, Apple may decide to introduce their own Bone Conduction product to the market. What’s interesting is if Facebook does release any of these products, they won’t actually be in direct competition with Apple should they too decide to release even a simple pair of Bone Conduction headphones.
At this point, this is because Facebook’s devices are primarily AR, VR, or a pair of Smartglasses that simply makes use of Bone Conduction technology to deliver one aspect of its sound, while Apple’s device would likely be a pair of Bone Conduction headphones, specifically designed to listen to music and audio while on the go, using Bone Conduction technology as it’s unique selling point. And competing with niche brands like AfterShokz.
From a business perspective, there’s is a trend from other companies like Bose who are selling Bose Frames / Audio Sunglasses that have speakers built-in and it is likely at some point depending on the technology included in these glasses consumers are going to start walking around town listening to music and audiobooks through their AR glasses!
In fact, Facebook’s device may actually work to Apple’s benefit. We’ve speculated for a while that Apple is waiting for Bone Conduction technology to enter the public consciousness more before they bring a product to market. Besides the fact that they may not have their technology up to Apple standards. This may be one of the reasons why they haven’t released one as of yet.
Facebook’s AR Smart glasses may give Hybrid Audio Systems with Air and Bone Conduction technology the exposure it needs to pique the general public’s curiosity. Then allowing Apple to unveil its own product and change the audio landscape yet again.
Smart Glasses
The ultimate goal of Smartglasses is to have a user-friendly wearable interface that the public will purchase and wear for different applications. With the rate of design and technology there are basic pairs of Smartglasses like the Vue Smartglasses that are already on the market and have open ear audio, built-in microphone, and touch controls so you can check the weather, call someone, and send a text message by Bluetooth with support from Siri, Google Assistant, and Alexa.
Then you have a new product like Bosch’s Sensortec’s innovative Smartglasses Light Drive that can integrate into any glasses.
“The Bosch Sensortec’s Smartglasses Light Drive is a microelectromechanical system (MEMS) based collimated light scanner inside the Bosch Smartglasses Light Drive Module that scans a holographic element (HOE) that is embedded in the lens of the smartglasses. This HOE redirects the light beam onto the human retina surface, directly painting a picture that is always in focus.”
Bosch Smartglasses Light Drive Module
According to Wikipedia Smartglasses:
“Superimposing information onto a field of view is achieved through an optical head-mounted display (OHMD) or embedded wireless glasses with transparent heads-up display (HUD) or augmented reality (AR) overlay. These systems have the capability to reflect projected digital images as well as allow the user to see through it or see better with it.
Wikipedia Smartglasses
While early models can perform basic tasks, such as serving as a front end display for a remote system, as in the case of smartglasses utilizing cellular technology or Wi-Fi, modern smart glasses are effectively wearable computers which can run self-contained mobile apps. Some are hands-free and can communicate with the Internet via natural language voice commands, while others use touch buttons.”
Final Thoughts
The Facebook patents (coupled with the Apple patents) are very telling. While we may be several years away from the release of Facebook’s AR glasses, or they may never be released at all. They do give us a window into the minds of the engineers in both companies. It also looks as though Facebook Technologies has committed a sizable amount of funding and resources to these designs. This in turn allows us to make our predictions about future technology and trends that we may soon see.
We know Bone Conduction headphones technology is constantly improving, as seen in the Facebook Hybrid Audio Systems. The problem is we don’t know when exactly. But the release of any of Facebook’s patents above could start a domino effect where more and more technology giants try their hand at Bone Conduction or Hybrid Audio Systems technology.
Like they have so many times before, our money is on Apple being at the forefront of any product introduction that happens, but we also think niche companies like AfterShokz with their new OptiShokz and Bosch may also do well out of such a trend. After all, they’ve been doing it a lot longer than Apple has.
References:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4359134/
https://www.audiologyonline.com/articles/providing-some-direction-beamformers-explained-26260
https://www.aao.org/eye-health/glasses-contacts/learn-nine-essential-parts-of-eyeglasses