Apple Seriously Invests in AR... Developing Sketchbook App for iPad

Patently Apple, Cited by the US Patent Office Report

[Asia Economy Reporter Minyoung Cha] Apple, leading the development of augmented reality (AR) technology, is developing an iPad sketchbook application (app) that makes it easy to create composites of computer graphics (CG) work and real-life photos.

IT specialized media Patently Apple reported on the 12th (local time) that "the United States Patent and Trademark Office has published a CG-related patent filed by Apple." The disclosed patent technology was filed on August 22 (local time).

The disclosed patent technology includes methods and systems for placing CG graphic objects based on sketches through the iPad. The targets to which the graphic objects can be applied range widely from virtual environments to extended reality (XR) environments.

The patent technology involves inputting a content creation interface using tools such as a 'sketchpad,' creating graphic objects using 3D models, and then placing photos (images) captured through a camera together with the CG work.

Apple plans to utilize see-through technologies used in the AR industry, such as Optical See-Through and Video See-Through methods, to ensure that graphics are naturally overlaid on real-life photos during the compositing process. For example, when compositing a palm tree graphic onto a real photo of a table, parts of the table that are obscured will be naturally transparent.

Apple has consistently invested in research and development (R&D) to introduce app development tools applying AR technology. A representative example is 'ARKit 5,' an AR app development tool. Compared to previous versions, ARKit 5 has enhanced motion tracking capabilities and supports face tracking through the iPad Pro's ultra-wide camera. By providing development tools, Apple can attract more AR apps. 'XR headsets' and 'smart glasses' are also planned to be introduced sequentially around next year and 2025.

Apple CEO Tim Cook has emphasized the importance of AR technology connected to reality for several years. In a 2016 interview with CNET, Cook said, "AR and mixed reality (MR) will be important keys to providing information at the right place and time," adding, "VR separates users from the real world, but users do not want to be away from the real world for long."