My first AR app on iOS: Juego de la Rana

I can’t believe it’s September! That means summertime has just about passed, and the rainy season is setting in, but also that means we’re 3/4 of the way through our (current) one year visa to stay in the País Vasco 😥. We’ve made the most of it, though, recently having been in central Bilbao for a long nine days of fiestas for Aste Nagusia, followed by a more tranquil weekend in Mundaka alongside the Urdaibai Biosphere. Euskadi has proven to be a more beautiful countryside with a richer culture than even we expected, and if we only get to spend this one year here, it will be well worth it.

Since it’s been a while since my last technical post, the rest of this blog will focus on my most recent project. Those who follow my LinkedIn or Instagram pages may have seen a few recent posts on “Juego de la Rana,” an augmented reality (AR) app that I’ve been developing for iOS using the RealityKit framework. Now, you can even play it yourself!

La Rana Image Gallery

“Juego de la Rana” (“Igel Jokoa” in Euskera, or simply “Frog Game”) involves trying to throw a metal disk into the mouth of a brass frog statue. The statue sits on top of a walled table that is typically covered with a carpet or astroturf surface.

The first image above is the version of that game board that I modeled and rendered in Blender, before turning into a .usdz file to be imported into an iOS app. The frog statue itself, seen in the second image, is a 3D model that I created using Photogrammetry. The source of that particular model is a game that was outside of a konpartsa on the final day of Aste Nagusia (more on that in the video below). I was extremely impressed by how well the Photogrammetry method picked up the detail of the statue!

The third image is the icon I’ve attached to the game, and the last is the game itself, played from the comfort of my couch. Some late breaking details I added to the released version are the messages that appear to indicate success or failure of your latest toss, as well as fireworks when you get one of your disks into the frog’s mouth. If the animation included audio, you’d also hear sound effects associated with the appearance of the game board, tossing coins, muffled thuds when a disk strikes the board, and metallic clinks when it strikes the statue.

Making Of Video

I also made a YouTube video showing the “making of” process. The video is now a couple of weeks old, but still pretty cool. You can see here a bit of “real world” gameplay, at the same game board that I patterned the 3D model from, and the photos that I used in the photogrammetry. This also mentions Reality Composer, the Apple software used to prepare 3D models for their AR as well as extended Reality (XR) platforms. Whats XR, you ask? Well, among other things, it is the immersive environment provided by the Vision Pro headsets that Apple released last year. Reality Composer will be an enabler to get La Rana running as a fully immersive app on Vision OS, if I choose in the future.

La Rana Around Town

One of my recent hobbies (much to Rachel’s annoyance) is to pull out my phone and play La Rana while we tour some of the cool spots around Bilbao and close by. I think its pretty cool to bring a digital model into the real world like this. Its also one of the selling points I make for why I think folks, especially locals around the Basque country, might like the game. Normally, a Juego de la Rana board is fixed to a space at some tavern or elsewhere where customers entertain themselves with it while enjoying their beverage. With the app, you can bring it anywhere!

Here is a description of the locations in the gallery above:

• Teatro Arriaga: One of the most famous historic buildings in central Bilbao, just across the bridge from the train station, and marking the entrance to the Casco Viejo.
• El Triangulo: A tiny plaza inside the Casco Viejo surrounded by a grouping of bars and restaurants. You can see a real game of La Rana happening directly behind my digital version!
• Plentzia: A beach town that is the last stop on the Bilbao metro. There is a trail to a cliff overlooking the beach on the eastern side.
• Torre Iberdrola: The tallest building in Bilbao, completed in 2011. Just to its south is the Parque Doña Casilda Iturrizar.

Virtual Reality Mode

When I originally made this post, I had included a beta test link, which is now replaced with the “real” link to the App Store. This was because the initial build was still in review. That build was initially rejected, but for good reason! So, Apple’s policy is if you build an app that uses a hardware feature that requires a user permission, in this case the camera, that app must still have some functionality even if the user rejects the permission. At first, Juego de la Rana would only show a black background under that circumstance, which is no fun. To meet this requirement, I created a Virtual Reality (VR) format version of the game, which you can toggle to/from using the slider at the top. This VR version uses a scene of El Triangulo that I mocked up in Blender. Now, you can play even in places where you don’t want to or can’t use your camera.

This should also make for a good scene for the “Immersive View” mode on VisionOS, which I’ll create in the future.

Wrapping Up

Making “Juego de la Rana” has been a pretty cool project for me. It also ticks an item off the box for my desire to learn the AR toolkits that are embedded in modern iOS. All told, I’m just about exactly one month into its development, having had the idea to create a photogrammetry model of the frog statue on the very last day of Aste Nagusia, the 25th of August. So, not only am I impressed by the RealityKit technology, but it proved to not have an overwhelming barrier to entry.