Apple Vision Pro lets you turn any surface into a touchscreen for apps

With the visionOS DevKit available for developers to create Apple Vision Pro apps, we’re discovering more capabilities that this space computer will bring when it’s released in early 2024. Developer Steve Troughton-Smith sharing some of these capabilities, including the ability to flatten a visionOS window on a desktop.

With that, Apple Vision Pro apps can look and feel more realistic when performing some tasks, such as playing a song, typing on the keyboard, and more. Interestingly enough, Apple filed a patent in 2016, which was granted in 2020, about this technology that is being used with the Apple Vision Pro.

Although the link to the patent itself is now broken, AppleInsider it covered at the time. As reported by the publication, the patent states that “a natural way for humans to interact with (real) objects is to touch them with their hands. [Screens] that allow touch detection and localization on their surface are commonly known as touchscreens and are a common part of, e.g., smartphones and tablet computers today.”

While the best option would be to “put a sensor that can detect touch on the object or the human body,” Apple scientists thought that cameras could help track the user’s fingers to an extent – which is precisely what Vision Pro does.

Combining information from the patent with the preview given by Troughton-Smith using visionOS shows that interacting with virtual objects can be very realistic if we have a surface.

Not only will the app stay on the surface, but it will be as natural as approaching a book on a table or a keyboard on a surface.

While we’ll still have to wait a few months for the Apple Vision Pro release, we’re sure to learn a lot more about Apple’s first space computer now that the SDK and visionOS are out in beta.