Augmented Reality [AR] is a technology used to blend virtual elements with real world. Though it has been around for about three decades, AR technology’s growth has just been phenomenal in the recent years, as top brands and enterprises began to use AR to give consumers an experience like no other. The AR experience, quite rightfully, is called an immersive experience.
Apple’s cutting-edge Augmented Reality technology – the ARKit – enables you to bring this stunning and captivating technology right into your smartphones and iPads, where you can bring in 3D elements and seamlessly superimpose them into the physical world, reorient your device to view the virtual objects from different angles, and even interact with those virtual objects using gestures and movements.
Since AR is an advanced technology, and still in the expansion stage, it does come with certain limitations with respect to working on certain devices. As an iOS developer, before you even begin to think of developing an AR-based app, it is imperative to know on what devices the technology is compatible with.
Apple’s ARKit requires iOS devices with iOS 11 and A9 processor or later.
As of this writing, the ARKit works with the following devices:
iPhoneX, iPhone8, iPhone 8 Plus, iPhone 7, iPhone 7 Plus, iPhone 6s, iPhone 6s Plus, iPhone SE
iPad Pro (all models)
iPad 5 (5th Generation)
When it comes to designing Augmented Reality applications, Apple has set certain guidelines for its ARKit users to deliver constructive and optimum AR experience.
Let’s get started.
As AR app building is a long and detailed process, in this article we will focus on some of the prime design guidelines.
When Augmented Reality is Initiated
Everytime your app enters Augmented Reality, initialization process begins and it starts to detect your surrounding and surfaces. Since the speed of the detection varies on several factors, developers should inform users that the app is detecting, and to speed up the process they should encourage users to move around slowly scanning their surrounding.
Remember, AR is new for many users, and they have no idea how it functions. This kind of prompting will help users to know what is happening and what the app is doing.
Placing Virtual Elements
You should help users understand how to locate a surface and add an object. A visual indicator is a great way to inform users that the surface targeting mode is active. A trapezoidal reticle is an effective way to inform users to find a vertical or horizontal surface. This is would be called a surface detection indicator, and it would be like this:
Surface detection indicator
Once the surface is targeted, the indicator should change to tell them that object placement is now possible.
Object placement indicator
If the indicator’s orientation matches the alignment of the detected surface, it can help people foresee how the placed object will be aligned. Now, design an app specific visual indicator to tell that.
App specific indicator
Maintaining accuracy is important here, and respond appropriately when users place the object. If they tap the screen to place an object, place it right away by using information that is currently available. Once the surface detection is complete, subtly refine the object’s position. In case if the placed object is flowing out of the boundary of the detected surface, subtly adjust the object back onto the detected boundary.
User Interaction of Virtual Objects
Don’t make users interact with the object through separate controls, let them directly touch the object and control them. Like this:
It is more immersive to allow users to touch and move the object, but make sure that direct manipulation is smooth and not confusing. Use standard and familiar gestures.
Here’s the sample code for 3D interaction and UI controls in AR
Common gestures are widely meant for two-dimensional interface. Since AR is three-dimensional, developers need to consider certain approaches to simplify user interaction related to virtual objects.
Users might find it difficult to touch specific point of the objects, especially when the objects are either small or thin. So, when the app detects a gesture, it is best to assume that they want to change affect the object. Make it respond to gestures within a reasonable proximity.
Be wary of similar gestures. For example, the two-finger pinch gesture and two-finger rotation gesture.
Breathing Life Into 2D Images Through ARKit
Through the Apple’s ARKit, developers can enable apps to detect known 2D images in user’s real world. And by using the position of these images, virtual objects can be placed.
For example, an app recognizing a sci-fi film poster and then virtual aliens or spaceships can be made to fly around.
Here a sample code from Apple and the steps to enable image detection.
Handling AR Interruptions
1. The Arkit cannot track device position and orientation when AR is not active. One way to avoid interruption is to allow users to adjust virtual objects and settings within the AR experience. For instance, if a user is using the app to buy screens for windows and doors (with different colors), allow them to change colors without exiting AR.
2. The ARKit will not track device positioning and orientation during interruption too, like temporarily switching to some other app or taking a phone call. So, when the user comes back to AR after interruption, the previously placed virtual objects will show up in the wrong physical world position. One way to recover from these interruptions is to use relocalization. By enabling relocalization, the ARKit attempts to recover relevant information needed to restore virtual objects to their original positions.
Here’s a developer guidance on relocalization
Handling AR Challenges
1. There may be times when AR does not meet user expectations. At those instances, it is important not to force users to wait or struggle with object placement. Enable them to start over again till they see better results.
2. Teach users how to fix issues. If problems occur, state those problems and then offer possible suggestions to fix those. Like the below:
3. Make your AR app available only to supported devices. If AR works as a primary feature in your app make it available only to devices that support ARKit. If the app has AR as a secondary feature, and when users attempt to enter AR in an unsupported device, avoid displaying an error message (don’t offer optional AR feature in the first place).
A Glyph is nothing but an elemental symbol. Apple’s ARKit has its own copyrighted Glyph.
- This Glyph should be used strictly for initiating ARKit-based experience.
- In no case it should be altered, other than changing its size and color.
- It should not be used for other AR experiences which are not created using ARKit
- It should have a minimum amount of clear space of 10% of the Glyph’s height, and don’t let other element encroach this space
The clear space around the Glyph
You can download the Glyph from Apple Resources
Badging is the process of the identifying specific items that can be viewed in AR in ARKit, especially in apps that use an extensive collection of products. For example, if you are building an office furniture app, you can badge to mark certain furniture which users can preview before purchasing. The AR badge is to indicate that specific product can be viewed with AR in their real world surroundings.
See how they are badged with the symbol (top left on each furniture).
Apple offers two types of badges:
If you have space constraint, the Glyph-only badge.
* Do not change the color of the badge, nor use them for other purposes with AR experiences that are not created with Apple’s ARKit.
* If all objects in your app offer AR facility, there no point in using the badge. Use it only to distinguish between AR and Non-AR objects.
* The badge placement should be consistent throughout your app. It looks best when they are placed on one corner of the object’s photo. However, if you are placing it anywhere else, let it be consistent on all other object’s photos.
* The required minimum clear space around the badge is same as Glyph – i.e, 10% of badge’s height.
The AR Badges are available in collapsed and expanded form. You can download the Badges here.
Before we wrap, here are some tips for delivering a truly immersive experience.
1. Use the entire display area of your device to provide optimum AR experience
2. Do not add too many controls or information on the display area as it could spoil the experience
3. Though not all AR require realistic virtual objects, however, for best results, it is recommended that you create realistic 3D images and textures
4. Consider small spaces or places where there isn’t much place to move around while creating AR
5. Don’t encourage movement right away. First, give them time to adapt to the AR experience, then gradually encourage movement – especially in an AR game
6. Consider user safety
7. Use music and haptic (sound effect/bump) to make the AR experience more life-like
8. Since AR is an advanced technology, use simple and friendly conversation within your app to help people understand better
As mentioned earlier, Augmented Reality is new to mobile apps. The technology is expanding, and so are the guidelines. We hope the above mentioned guidelines have helped to gain a better understanding of what you should and should do when designing for AR. Apple’s developer portal and Human Interface Guidelines are the best place to start, and they offer a wealth of information pertaining to design guidelines.