AN ARTICLE BY SANKET PRABHU (SOFTWARE ENGINEER, AUGMENTED REALITY, MOBILIYA)
Apple’s new reality-bending developer platform
In an interview to The Independent ea
rlier this year, Cook said that “the smartphone is for everyone, we don’t have to think the iPhone is about a certain demographic, or country or vertical market: it’s for everyone. I think AR is that big, it’s huge. I get excited because of the things that could be done that could improve a lot of lives”.
At this year’s Worldwide Developers Conference (WWDC-2017), Apple announced their first augmented reality software development kit, ARKit AR-SDK and offers developers the chance to create digital content on top of real world scenery on iOS 11 for iPhone and iPad through what Apple refers to as “World Tracking”.
Apple Vice President Craig Federighi announced ARKit at 2017’s WWDC keynote. The project contains a developer toolset that it will make available to nearly instantaneously make the iPhone and iPad the largest AR platform in the world, according to the company.
What is ARKit?
ARKit allows developers to tap into the latest computer vision technologies to build detailed and compelling virtual content on top of real-world scenes for interactive gaming, immersive shopping experiences, industrial design and more.
ARKit is Apple’s new ‘Developer Platform‘, allowing programmers to build apps that blend the real world with augmented digital objects. ARKit will come to compatible devices as part of iOS 11, which Apple is set to release in the fall.
- Game Engines: Unity, Unreal
- X-Code Library: SceneKit, MetaAPI
- Devices: iPhone SE, iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPad Pro, 9.7-inch iPad 2017
- Fast, stable motion tracking
- Plane estimation with basic boundaries
- Ambient lighting estimation
- Scale Estimations.
- SLAM tracking and sensor fusion
Since ARKit will be available on iPads and iPhones, Apple CEO Tim Cook claims it will represent the world’s largest AR platform. Compare it to Google Tango, for example, Google doesn’t automatically include Tango on every Android device; instead, manufacturers must choose to include it. In short, ARKit is the framework for developers to create crazy cool augmented reality applications.
Augmented reality is huge, and it’s only getting bigger
- Apple has invested heavily in augmented reality in recent years, making several VR/AR company acquisitions including PrimeSense, FaceShift, and Metaio.
- Apple’s much-anticipated “iPhone 8”, which is expected to be announced in September, is rumored to have 3D sensing capabilities, with augmented reality being one possible use for the technology.
- Basically, Apple is betting on developers wanting to build AR experiences for the millions of iOS users out there
Features offered by ARKit
By using the Built-in camera, Processors, and Motion Sensors found in iOS devices, virtual content appears on top of real-world scenes, and users don’t need any special equipment to enjoy them because ARKit does the heavy lifting.
Other AR software often requires some sort of physical tracking mechanism (like an ImageTarget Or QR code), which inevitably becomes a major piece of friction if you are trying to get anyone to use this stuff.
Another incredible aspect of ARKit is how it handles lighting adjustments in real time
ARKit comes with sophisticated motion tracking, which Apple called “fast and stable motion tracking.” This will ultimately allow for a more immersive experience, as objects will appear to be intrinsically part of the real space.
It will take some time to go through ARKit’s full potential, but Apple demoed a couple AR experiences created with the tech to show a bit of what’s possible.
In a demonstration of software produced by the new ARKit, the software identified a table surface and applied a virtual coffee cup properly scaled to the surface. Following a lamp’s addition to the surface, the lighting model adjusted dynamically as Federighi moved the lamp around the cup.
Apple ARKit Demo: Dynamic Lighting
How ARKit Works
The valuable information available on Apple’s ARKit developer page sheds more light on this platform. we can categorize the working of ARKit in following aspects.
- Visual-Inertial Odometry (VIO):which will let ARKit track the world, and for this, it will fuse camera sensor data with CoreMotion data. According to Apple, “these two inputs allow the device to sense how it moves within a room with a high degree of accuracy, and without any additional calibration.”
- Scene Understanding and Lighting Estimation:ARKit analyses the scene based on the camera view, and then find “horizontal planes like tables and floors, and can track and place objects on smaller feature points as well.” The ARKit also relies on the camera sensor to figure out the amount of light in a room, and will then apply the correct light to the virtual objects.
- High-Performance Hardware and Rendering Optimizations: ARKit runs on the Apple A9 and A10 processors. These processors deliver breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes. You can take advantage of the optimizations for ARKit in Metal, SceneKit, and third-party tools like Unity and Unreal Engine.
Supporting Hardware & Technology
ARKit takes advantage of the iPhone’s camera, motion sensors, and graphics processors to allow developers create apps and games that combine the real world with the virtual world. From developing to run AR app using ARKit following are the prerequisite:
To run on the device:
- Operating System: iOS 11
- Hardware Support: A9 or A10 chip Only.
These include: iPhone SE, iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPad Pro, 9.7-inch iPad 2017
- X-Code Version:X-Code 9
- AR SDK: ARKit SDK
- Special Plugins: SceneKit or SpriteKit
- Game Engine: Unity 3D & Unity ARKit Plugin (Optional)
- Apple devices with only hardware with the A9 or A10 familyprocessors able to use the augmented reality software. which means that the iPhone 6s, iPhone SE, and iPhone 7 families can use the technology.
- For iPads, the situation is a bit morerestrictive with the 2017 iPad, and the iPad Pros the only compatible models.
- Apple like tech giant is now investing in and promoting AR validates the market that other AR SDK like Vuforia, Wikitude has been working hard to create for nearly a decade.
Without a doubt, Apple’s announcement will contribute to boosting AR usage to an unprecedented worldwide scale and we are looking forward to what lays ahead.
With the release of ARKit to developers, Apple is bringing augmented reality to the masses on iPhones, iPads. In comparison, Microsoft’s HoloLens is still a far more expensive. Rival Google’s Project Tango, a specialized AR platform, is available on a handful of devices. Apple isn’t even asking to invest in a special hardware to experience augmented reality apps instead, it is pushing developers to create the apps.
With this release, Apple has officially joined the rising AR scene, alongside Google and Facebook, which have also unveiled augmented reality tools of their own. This means that the large user base of iPhone users, all with access to cutting edge augmented reality software, means it’s inevitable that we’re going to some interesting iOS exclusives in the coming months.