Apple wins a patent for Spatial Audio Navigation providing an indoor turn

May 25, 2024

Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to spatial audio navigation being available on future AirPods, smartglasses and a lighter-weight version of Vision Pro. The system uses directionality of audio played through a binaural audio device to provide navigational cues to the user to find their way through a mall, other venue or city park. The system could also provide audio navigation to a driver of a vehicle.

Spatial Audio Navigation

Apple's granted patent covers methods and apparatus for spatial audio navigation. Embodiments of a spatial audio navigation system and spatial audio navigation methods are described that may, for example, be implemented primarily using smartglasses or an HMD (Vision Pro) as the patent figures presented below illustrate. The spatial audio navigation system provides navigational information in audio form to direct users to destinations such as real world locations, people, or objects. Instead of using vocal directions or relying on visual cues as in conventional navigation applications, the spatial audio navigation system uses directionality and distance of audio played through a binaural audio device (e.g., headphones, headsets, wired or wireless earbuds, etc., collectively referred to as a “headset”) to provide navigational cues to the user.

Embodiments of the spatial audio navigation system and spatial audio navigation methods as described herein instead use directionality and distance of sound played through a binaural audio device to provide navigational cues to the user. Embodiments thus use the spatial location of the apparent source of a sound to guide the user in a certain direction. For example, to guide the user on a path while listening to music, the apparent source of the music may be placed in front of the user to guide the user along the path, and moved to the side of the user to prompt the user to make a turn on the path. In this way, the music is not interrupted, and embodiments provide a subtler method to convey navigational information than conventional navigation applications.

The same can be done with other audio sources including but not limited to audio books, telephone conversations, simulated or captured ambient noise, simulated sounds such as tones, bells, sirens, or white noise, or recorded sounds such as the sound of a railroad train or a pack of wolves.

In embodiments of the spatial audio navigation system, a current location, target location, and map information may be input to pathfinding algorithms to determine a real world path between the user's current location and the target location. The spatial audio navigation system may then use directional audio played through the headset to guide the user on the path from the current location to the target location.

While the patent describes spatial audio navigation system implemented in a mobile multipurpose devices like AirPods, embodiments of a spatial audio navigation system may also be implemented in VR/MR systems implemented as head-mounted displays (HMDs) that include location technology, head orientation technology, and binaural audio output; speakers integrated in the HMD may be used as the binaural audio device, or alternatively an external headset may be used as the binaural audio device.

Apple's patent FIG. 1 illustrates spatial audio navigation using an iPhone and AirPods; in FIG. 2A, Apple illustrates that spatial audio navigation could be used with future smartglasses or a more advanced Vision Pro headset that users will be able to easily use at a mall or elsewhere.

Apple's granted patent Figs. 5C and 5G illustrates various spatial audio navigation methods that may be used by a spatial audio navigation system. Apple's patent FIG. 6C below illustrates a discrete path tracing method in which a sound is moved in discrete intervals along the path. In some embodiments, when the sound reaches the end of the path, it returns to the beginning of the path and the process repeats.

For more details, review Apple's granted patent 11709068. The foundation of this invention was covered in a Patently Apple report posted back in 2020.

Posted by Jack Purcher on July 25, 2023 at 07:21 AM in 2. Granted Patents, Audio, HomePod, AirPods, AirPods Max, HMDs, Apple Vision Pro, Smartglasses +, Maps, Indoor Location | Permalink | Comments (0)