So the last few weeks I’ve been busy finalising my DVA exegesis, but also experimenting with the new augmented reality ARKit Quicklook functionality that’s embedded in iOS12. Essentially, if you load up a USDZ file (a new file format created by Apple and Pixar) you can launch that file as a 3D virtual object. It works through the standard iOS apps such as Safari, Mail, and Files. No coding and no special apps required!

If you want to try out the Riots Not Diets chrome sculpture yourself and are using Safari on iOS12, you can view the USDZ file here. A cube icon will appear on the top right hand of the image if your device is capable of viewing USDZ files. Click it to open up AR Quicklook, then select ‘AR’ and help the camera find a flat surface. You can move the object around, scale it, and rotate it.

The handling of the files is really fantastic. It has a tonne of built-in features such as automatic horizontal plane detection, a nice transparent overlay when it’s searching for a plane to launch on, vibration of your iPhone when it’s placing the object, built in scale, rotate, and positioning, built in shadows, and my favourite, built in dynamic reflections from the physical scene. I’ve had heaps of fun with mirrored surfaces (see the above video for a couple of examples). If you want to know more about this watch the Apple Keynote here. If you want to try out some of Apple’s examples shown in the keynote, visit this page using Safari on iOS 12.

It has some limitations which I list below. Hopefully this is helpful for anyone experimenting with this emerging technology.

Generating a USDZ file isn’t easy (yet)

This is the biggest hurdle so far. The only way to do this on the mac is via the terminal and some Xcode commands.

The only other way I’ve found is through the online 3D app Vectary. You can export 3D objects straight to USDZ — they email you a link when it’s complied. The examples above were created this way, I generated vector graphics in Illustrator, then extruded these in Vectary. The program works well, and it’s also free, which is a massive bonus.

Animation is even harder, and from what I’ve read I’m not even going to attempt it yet.

I’m eagerly awaiting what Adobe comes up with in their AR-specific Project Aero software. I was hoping Dimension 2019 would be able to export to USDZ, but unfortunately that app still seems to be horribly crippled (a shame, as it’s easy to use for non-tech designers like me).

Quicklook has limited distance and scale

When you enter AR mode in Quicklook, the camera focus seems to be limited to only a few meters. You can place an object close to you and then move it further away, but it will stop moving when you hit an artificial barrier around around 20 meters away. If you try to walk further away, the object just starts to clip.

This is a bit of a shame, as my first reaction was to generate house-sized type and start dropping them all over the place. I tried doing this from an elevated walkway onto a park below, but because of the distance it wouldn’t allow me to drop the object. I could drop the object at my feet and then move it further, but it became incredibly unstable and had trouble tracking.

I also tried projecting on water, which did work, but after a while it got confused by the water movement and the object shuttled away (video of this on my Instagram).

Automatically grounds floating objects

No matter how your USDZ file is set up, Quicklook will automatically ‘ground’ it to the plane (this seems to be a recent change, as the Keynote discussed having to do this manually with your object). This is fine for 99% of people, but because I like floating graphics, it’s a pain. My workaround has been to add in a tiny element at ‘ground’ level, with the main object floating above this.

Needs plenty of light

This isn’t a surprise, but it needs a fair bit of light to detect the surface plane. Helpfully, it has a pop up if your scene is too dark.

Limited video recording

As you have to screen record your augmentations, you don’t get the full video quality that the iPhone is capable of unfortunately (this seems to be true for all AR though).

No automatic vertical plane detection (yet)

I’m hoping this will come shortly, just like when vertical plane detection was added later in ARKit 1.5.

Web hosts don’t know how to handle these files (yet)

Even though Safari will automatically run the file in an AR space, you need to set up your web server so it recognises USDZ files first. This is done by setting up custom MIME type in your CPanel backend. Here is a great write up on how to do this.

Update: it can be buggy

I guess this is to be expected due to the emerging nature of the technology, but after using Quicklook USDZ files in my exhibition, have noticed it can sometimes freak out. Around 5% of the time you get an error (Universal Scene Description (Mobile)) and it won’t load at all. Cancelling out and reloading fixes it. This seems to happen with complex and simple objects, so need to do some more testing to work out what is going on…