Initial thoughts on Apple’s new ARKit Quicklook and USDZ file format

David SargentAR Quicklook, ARKit, Augmented Reality, DVA

So the last few weeks I’ve been busy finalising my DVA exegesis, but also experimenting with the new augmented reality ARKit Quicklook functionality that’s embedded in iOS12. Essentially, if you load up a USDZ file (a new file format created by Apple and Pixar) you can launch that file as a 3D virtual object. It works through the standard iOS apps such as Safari, Mail, and Files. No coding and no special apps required!

If you want to try out the Riots Not Diets chrome sculpture yourself and are using Safari on iOS12, you can view the USDZ file here. A cube icon will appear on the top right hand of the image if your device is capable of viewing USDZ files. Click it to open up AR Quicklook, then select ‘AR’ and help the camera find a flat surface. You can move the object around, scale it, and rotate it.

The handling of the files is really fantastic. It has a tonne of built-in features such as automatic horizontal plane detection, a nice transparent overlay when it’s searching for a plane to launch on, vibration of your iPhone when it’s placing the object, built in scale, rotate, and positioning, built in shadows, and my favourite, built in dynamic reflections from the physical scene. I’ve had heaps of fun with mirrored surfaces (see the above video for a couple of examples). If you want to know more about this watch the Apple Keynote here. If you want to try out some of Apple’s examples shown in the keynote, visit this page using Safari on iOS 12.

It has some limitations which I list below. Hopefully this is helpful for anyone experimenting with this emerging technology.

Generating a USDZ file isn’t easy (yet)

This is the biggest hurdle so far. The only way to do this on the mac is via the terminal and some Xcode commands.

The only other way I’ve found is through the online 3D app Vectary. You can export 3D objects straight to USDZ — they email you a link when it’s complied. The examples above were created this way, I generated vector graphics in Illustrator, then extruded these in Vectary. The program works well, and it’s also free, which is a massive bonus.

Animation is even harder, and from what I’ve read I’m not even going to attempt it yet.

I’m eagerly awaiting what Adobe comes up with in their AR-specific Project Aero software. I was hoping Dimension 2019 would be able to export to USDZ, but unfortunately that app still seems to be horribly crippled (a shame, as it’s easy to use for non-tech designers like me).

Quicklook has limited distance and scale

When you enter AR mode in Quicklook, the camera focus seems to be limited to only a few meters. You can place an object close to you and then move it further away, but it will stop moving when you hit an artificial barrier around around 20 meters away. If you try to walk further away, the object just starts to clip.

This is a bit of a shame, as my first reaction was to generate house-sized type and start dropping them all over the place. I tried doing this from an elevated walkway onto a park below, but because of the distance it wouldn’t allow me to drop the object. I could drop the object at my feet and then move it further, but it became incredibly unstable and had trouble tracking.

I also tried projecting on water, which did work, but after a while it got confused by the water movement and the object shuttled away (video of this on my Instagram).

Automatically grounds floating objects

No matter how your USDZ file is set up, Quicklook will automatically ‘ground’ it to the plane (this seems to be a recent change, as the Keynote discussed having to do this manually with your object). This is fine for 99% of people, but because I like floating graphics, it’s a pain. My workaround has been to add in a tiny element at ‘ground’ level, with the main object floating above this.

Needs plenty of light

This isn’t a surprise, but it needs a fair bit of light to detect the surface plane. Helpfully, it has a pop up if your scene is too dark.

Limited video recording

As you have to screen record your augmentations, you don’t get the full video quality that the iPhone is capable of unfortunately (this seems to be true for all AR though).

No automatic vertical plane detection (yet)

I’m hoping this will come shortly, just like when vertical plane detection was added later in ARKit 1.5.

Web hosts don’t know how to handle these files (yet)

Even though Safari will automatically run the file in an AR space, you need to set up your web server so it recognises USDZ files first. This is done by setting up custom MIME type in your CPanel backend. Here is a great write up on how to do this.

Update: it can be buggy

I guess this is to be expected due to the emerging nature of the technology, but after using Quicklook USDZ files in my exhibition, have noticed it can sometimes freak out. Around 5% of the time you get an error (Universal Scene Description (Mobile)) and it won’t load at all. Cancelling out and reloading fixes it. This seems to happen with complex and simple objects, so need to do some more testing to work out what is going on…

 

Augmented Reality Activism paper for CreateWorld 2017

David SargentActivism, Augmented Reality, Conferences, DVA

The paper I presented at the 2017 CreateWorld Conference, Repurposing Augmented Reality Browsers for Acts of Creative Subversion is now available.

It includes a bit of a ‘how to’ if you are interested in co-opting Augmented Reality Browsers for other purposes. It’s not complicated, but I did come across a few issues which might be of interest if you wanted to learn from my mistakes! Also, make sure you read my other post on strategies to try and protect your identity if you are planning to try this out.

Abstract:

Consumer facing Augmented Reality (AR) technology offers innovative new ways for consumers to engage and interact with brands and products via interactive advertising and experiences. Conversely, this technology also creates new channels that can be exploited and subverted by those who wish to generate critical reflection of consumerist culture. This paper aims to highlight that consumer AR technology presents new and unique opportunities for activists interested in subversive communication.
The above video is a bit of an outtake due to that car parking in exactly the wrong spot. One of the drawbacks of working in the public sphere! If you would like to read all of the papers from the conference, you can find them here (warning: 50MB PDF file).

 

 

36 Days of Type

David SargentLettering

Lasercut #J by @davidsargent_

A post shared by 36daysoftype (@36daysoftype) on

After three years of trying — finally had one of my letters featured by 36 Days of Type! Funnily enough, this letterform happened very quickly and turned out great! If you are interested, there is a process video of the laser cutter at work on my Instagram page.

 

Lost & Found mural for Brisbane City Council

David SargentLettering, Murals

Now that it’s all done and dusted, here are a couple of photos my mural Lost & Found, painted for the Brisbane City Council Brisbane Canvas program. In a few short years, this program has funded some fantastic art around town.

There are seven other walls being painted this year as part of the same program, including work by Sophia Mary Mac, Daniel Brook, Deb Mostert, Kirsten Baade, David Houghton, Carley Cornelissen, and James Alley.

I wrote about what I was trying to do with this work here.

 

#SubvertTheCity co-ordinates ad takeovers across the globe

David SargentActivism, Augmented Reality, Body Image, DVA, Lettering

I’ve been keenly watching the Subvertisers International for a little while now. They’ve been connecting different activist groups and individuals together in an attempt to push back against advertising which is ‘invading our public spaces and co-opting our media in an effort to promote the endless consumption of goods, services and brands’. 25 March 2018 was deemed #SubvertTheCity day, with co-ordinated activities happening all across the globe.

Most of these groups are contributing via a ‘take out advertising and replace it with something new’ tactic, with some fantastic results. Closer to home, I spotted this happening here in quiet little Brisbane, with a really fantastic culture jam of an ad for the defence force just down the road from me. There are some alternative actions taking place too — I really liked this example by AdBlock Bristol where advertising spaces were replaced with paper for passers by to contribute to.

My work in this area is also non-destructive, so I thought it would be a good time to contribute this type of intervention. The more I think about the future of augmented reality and what our world might look like in ten years time, the more I think pushing back via this kind of technology will be vital. I spent the day finding some suitable target ads around town and embedding my augmentations, including the one above.

Painting the Lost & Found mural in Teneriffe, Brisbane

David SargentLettering, Murals

Time lapse of my recent mural painted for Brisbane City Council as part of their Brisbane Canvas program. The mural will also be included in the 2018 Brisbane Street Art Festival. I wouldn’t consider myself a ‘street artist’, but I guess it’s art, and it’s on the street? Either way, the other people included are fantastic at what they do, so a great honour to be part of it.

People who paint large scale murals don’t get enough respect. It is HARD work. I learnt a whole lot of lessons.

Lost & Found is a phrase I really like as it’s so unusual when taken out of it’s usual context. When people saw it being painted on the wall I had so many questions about what it meant. My response was usually, ‘what does it mean to you?’, but that didn’t usually go down well! When I was putting the idea together it came from the dramatic urban renewal of Teneriffe. My experience with the neighbourhood was many visits to Paddy’s Markets as a child (btw this was the best place to get canvas school bags back then). It’s now a very different place. Interestingly, Teneriffe was literally ‘lost’ when the suburb was absorbed into Newstead in the 70s then ‘found’ when it was re-instated as a proper suburb again after becoming one of the most desired (and expensive) places to live in Brisbane. The design is inspired by the fantastic art deco architecture and typography that remains in the area.

The call out for design proposals happened at the perfect time for me as I had just attended a fantastic art deco lettering workshop by Nick Misani and this was a great opportunity to try all those new tricks out.

It’s up to the viewer to determine if Lost & Found is a positive or negative statement on this change. For me, especially after spending a week there, it’s a little bit of both. You tend to see the best and worst of a neighbourhood when you’re spending entire days hanging out there, it was actually a very interesting experience and a great way to get a feel for the location. The irony of commissioned public art (a typical feature of gentrification) is not lost on me!

The mural took on another meaning after we saw someone hit a dog with their car in the street right in front of us. They slowed and then quickly took off… the dog bolted off closely chased by my wonderful dog whispering partner who spent the afternoon trying to find him. He was lost for a few hours before being eventually found hiding at the end of the day. A vet visit revealed some scratches but no serious injury. We painted a small dog in the mural as a tribute to ‘Bowie’ and his Lost & Found adventure.