Augmented Reality Design Guidelines: Here’s What iOS Developers Need to Know
Augmented reality, the process of overlaying virtual elements into the real world background, as most believe, is not an all new technology. It was experimented decades ago and has been quietly mushrooming in its scope and importance.
In the last few years, the interest is augmented reality has been spiked up, thanks to the advancing immersive experiences in smartphones. Apple recently announced that all of the new iPhones will have faster A13 processors to accommodate and help users seamlessly use AR on their iPhones.
If Apple is bracing up Augmented reality, its not that hard to assume its impact in the coming future.
In this blog we will see some of the important guidelines and best practices that Apple has set down to help developers successfully bring AR into iPhones.
Since AR is an advanced technology, and still in the expansion stage, it does come with certain limitations with respect to working on certain devices. As an iOS developer, before you even begin to think of developing an AR-based app, it is imperative to know on what devices the technology is compatible with.
Apple’s ARKit device compatibility:
ARKit : iOS devices with iOS 11 and A9 processor or later. It works with the following devices.
iPhoneX, iPhone8, iPhone 8 Plus, iPhone 7, iPhone 7 Plus, iPhone 6s, iPhone 6s Plus, iPhone SE
iPad Pro (all models)
iPad 5 (5th Generation)
ARKit 3: The ARKit 3 is the latest offering from Apple and it provides some amazing features like People Occlusion where you can show AR content in front of or behind people.
iPhones: iPhone XS, iPhone XS Max, iPhone XR
iPads: 2018 iPads (11 inch & 12.9inch)
As AR app building is a long and detailed process, in this blog we will focus on some of the Apple’s prime AR design guidelines.
Let’s get started.
When Augmented Reality is Initiated
Every time your app enters Augmented Reality, initialization process begins and it starts to detect your surrounding and surfaces. Since the speed of the detection varies on several factors, developers should inform users that the app is detecting, and to speed up the process they should encourage users to move around slowly scanning their surrounding.
Remember, AR is new for many users, and they have no idea how it functions. This kind of prompting will help users to know what is happening and what the app is doing.
Placing Virtual Elements
You should help users understand how to locate a surface and add an object. A visual indicator is a great way to inform users that the surface targeting mode is active. A trapezoidal reticle is an effective way to inform users to find a vertical or horizontal surface. This is would be called a surface detection indicator, and it would be like this:
Once the surface is targeted, the indicator should change to tell them that object placement is now possible.
If the indicator’s orientation matches the alignment of the detected surface, it can help people foresee how the placed object will be aligned. Now, design an app specific visual indicator to tell that.
Maintaining accuracy is important here, and respond appropriately when users place the object. If they tap the screen to place an object, place it right away by using information that is currently available. Once the surface detection is complete, subtly refine the object’s position. In case if the placed object is flowing out of the boundary of the detected surface, subtly adjust the object back onto the detected boundary.
User Interaction of Virtual Objects
Don’t make users interact with the object through separate controls, let them directly touch the object and control them. Like this:
It is more immersive to allow users to touch and move the object, but make sure that direct manipulation is smooth and not confusing. Use standard and familiar gestures.
Here’s the sample code for 3D interaction and UI controls in AR
Common gestures are widely meant for two-dimensional interface. Since AR is three-dimensional, developers need to consider certain approaches to simplify user interaction related to virtual objects.
Users might find it difficult to touch specific point of the objects, especially when the objects are either small or thin. So, when the app detects a gesture, it is best to assume that they want to change affect the object. Make it respond to gestures within a reasonable proximity.
Be wary of similar gestures. For example, the two-finger pinch gesture and two-finger rotation gesture.
Breathing Life Into 2D Images Through ARKit
Through the Apple’s ARKit, developers can enable apps to detect known 2D images in user’s real world. And by using the position of these images, virtual objects can be placed.
For example, an app recognizing a sci-fi film poster and then virtual aliens or spaceships can be made to fly around.
Here a sample code from Apple and the steps to enable image detection.
Handling AR Interruptions
- The Arkit cannot track device position and orientation when AR is not active. One way to avoid interruption is to allow users to adjust virtual objects and settings within the AR experience. For instance, if a user is using the app to buy screens for windows and doors (with different colors), allow them to change colors without exiting AR.
2. The ARKit will not track device positioning and orientation during interruption too, like temporarily switching to some other app or taking a phone call. So, when the user comes back to AR after interruption, the previously placed virtual objects will show up in the wrong physical world position. One way to recover from these interruptions is to use re-localization. By enabling re-localization, the ARKit attempts to recover relevant information needed to restore virtual objects to their original positions.
Here’s a developer guidance on relocalization
Handling AR Challenges
- There may be times when AR does not meet user expectations. At those instances, it is important not to force users to wait or struggle with object placement. Enable them to start over again till they see better results.
2. Teach users how to fix issues. If problems occur, list those problems and then offer possible suggestions to fix those. Like the below:
3. Make your AR app available only to supported devices. If AR works as a primary feature in your app make it available only to devices that support ARKit. If the app has AR as a secondary feature, and when users attempt to enter AR in an unsupported device, avoid displaying an error message (don’t offer optional AR feature in the first place).
Guidelines for using AR Glyphs
- This Glyph should be used strictly for initiating ARKit-based experience
- In no case it should be altered, other than changing its size and color
- It should not be used for other AR experiences which are not created using ARKit
- It should have a minimum amount of clear space of 10% of the Glyph’s height, and don’t let other element encroach this space
You can download the Glyph from Apple Resources
Badging is the process of the identifying specific items that can be viewed in AR in ARKit, especially in apps that use an extensive collection of products. For example, if you are building an office furniture app, you can badge to mark certain furniture which users can preview before purchasing. The AR badge is to indicate that specific product can be viewed with AR in their real world surroundings.
Apple offers two types of badges:
If you have space constraint, the Glyph-only badge.
Guidelines of using Badges
- Do not change the color of the badge, nor use them for other purposes with AR experiences that are not created with Apple’s ARKit
- If all objects in your app offer AR facility, there no point in using the badge. Use it only to distinguish between AR and Non-AR objects
- The badge placement should be consistent throughout your app. It looks best when they are placed on one corner of the object’s photo. However, if you are placing it anywhere else, let it be consistent on all other object’s photos
- The required minimum clear space around the badge is same as Glyph – i.e, 10% of badge’s height
The AR Badges are available in collapsed and expanded form. You can download the Badges here.
Before we wrap, here are some tips for delivering a truly immersive experience.
- Use the entire display area of your device to provide optimum AR experience
- Do not add too many controls or information on the display area as it could spoil the experience
- Though not all AR require realistic virtual objects, however, for best results, it is recommended that you create realistic 3D images and textures
- Consider small spaces or places where there isn’t much place to move around while creating AR
- Don’t encourage movement right away. First, give them time to adapt to the AR experience, then gradually encourage movement – especially in an AR game
- Consider user safety
- Use music and haptic (sound effect/bump) to make the AR experience more life-like
- Since AR is an advanced technology, use simple and friendly conversation within your app to help people understand better
As mentioned earlier, Augmented Reality is new to mobile apps. The technology is expanding, and so are the guidelines. We hope the above mentioned guidelines have helped to gain a better understanding of what you should and should do when designing for AR. Apple’s developer portal and Human Interface Guidelines are the best place to start, and they offer a wealth of information pertaining to design guidelines.
Ideaplunge has been offering intuitive iOS solutions to Startups and Fortune 500 companies. With years of hands-on experience in crafting mobile apps for Retail, Healthcare, Social, Lifestyle and Fashion, Ideaplunge is one of the top iOS app development companies in Bangalore, with clients in over five countries across the globe.
Have a project in mind? Looking for iOS expertise for your project? Write to us at firstname.lastname@example.org