Adding to one of my common themes of late, a newfound hobby of mine is taking a zillion pictures of something interesting and turning it into a 3D model using XCode and RealityKit Photogrammetry. I then proceed to bring the model into Blender, where I clean up the mesh and textures to fix some of the data artifacts inherent to the 3D scanning process. In this case, I also spent some time learning the animation features in Blender, and subsequently generated the delightful video of a spinning plate of Gildas that you see below.
Where did I encounter such delicious Gildas?
If you read Rachel’s latest post, you would have seen the segment on Herri Kirolak, which we encountered at Bilbao Basque Fest during Semana Santa. The festival wasn’t all stone-lifting and hay-bale-throwing, however, it also included opportunities to sample the Basque gastronomy. We attended a Cata de Gildas (Gilda tasting, much like Cata de Vinos is a wine tasting), which for the uninitiated are a skewer with olives, piparras, and (usually) anchovies.
Our hosts for the Cata de Gildas were La Gilda del Norte, who I’ve come to find out are one of the primary suppliers of the ingredients to many of the bars in Bilbao. Now that I recognize the brand, I notice their delivery trucks arriving many mornings around the same time that I am walking to Spanish class.
From right-to-left in the image (the order that the leader of the cata suggested we eat them), the three Gildas we tasted were:
- Classic Gilda with antxoa (note the tx for the Basque spelling). The anchovy is cured in salt, giving it a darker color and saltier taste.
- Gilda with boquerón. The anchovy is marinated in vinegar, giving it a whiter color and milder taste.
- Gilda picante. Classic Gilda, with a red alegria pepper added to the mix, making it slightly spicy.
The latter is our go-to, as we prefer more spice than the average Spaniard. We also enjoy variations on the classic style, including Gildas with shrimp, octopus, or even queso.
The photo above is one of 48 that I took during the event, likely greatly confusing my neighbors. If you care to view the full set of images, I have uploaded it to Google Drive.
How did I generate the animation?
Most of the “hard work” is done by the RealityKit Photogrammetry module, which I execute from my own script implementation which I’ve posted on GitHub. To allow manipulation of the output, I specify the file format to be .obj, which can be imported into Blender.
I use Blender to perform corrections on the 3D model for artifacts that the Photogrammetry process misses. Often, these artifacts derive from the fact that you can never really capture every detail of the object from every angle, that it can be hard to distinguish parts of the object from the background, and from non-ideal lighting attributes.
Much of the cleanup process was similar to my second post about Puppy, such as using the texture clone tool to paste in colors that appeared more “correct” in regions where the photo capture was not ideal. For example, the model initially showed dark spots in regions of the plate that were shadowed in the photos. I corrected these to give them the same off-white color as neighboring regions on the plate.
The image below is actually clipped from the .usdz file viewer in XCode (not Blender), but is a good example of what I’m talking about. This depiction should be lighting-independent, meaning the objects shouldn’t cast directional shadows. The shadows we see in the lower right exist because they were there in the photos, but ideally shouldn’t exist in the 3D model. This is where I used the texture clone tool, using the “correct” color from neighboring sections to eliminate the shadow.
There were also issues with the mesh quality around some of the finer details at the edges of the plate, and the skewers. The plate is, of course, perfectly round, but since the edge of the plate is thin and the bottom in shadow, I got a number of these sections where it looked like someone or something took a bite out of it. Also, the skewers themselves were not nice and round and separated from the plate as you would expect, they kind of looked “melted” into the plate. I used sculpting tools in Blender to smooth out these surfaces, and ultimately replaced the skewers with primitive cylinders which I found to look better in the rendered version (smooth, better shadows, etc).
I implemented a form of three-point lighting in Blender, with the most powerful “key” light located near the camera, a less powerful “fill” light at a 90 degree angle to the left, and an even less powerful “back” light behind the plate. I tinkered with the powers, radius, and location of the lights until I got what I found to be about the right combination of light and shadow. Lighting is certainly an area that I could stand to improve though, as I still find the render to look a little bit cold.
Finally, I implemented the animation of the spinning plate, which is actually one of the first times I’ve attempted animation in Blender. This turned out to be fairly simple. I started by selecting my scanned object and the three objects I created, right-clicking, then selecting “insert keyframe” for location and rotation. Then, I toggled on the “auto keying” feature, slide to a later frame, and rotated to create a new keyframe. I repeated this until the plate had gone through a full 360 degree rotation. It turned out the hard part was getting the plate centered how I liked it, to help I placed a primitive cylinder in the model with radius approximately equal to the plate and centered at the origin, then translated the plate until it matched up with the cylinder. Then, I made sure to rotate about the origin, not the centroid of the object, which was biased away from the center of the plate due to the asymmetry of the scan.
With all that done, I ran the animation with the Cycles render engine, and with settings configured to export a 1024×1024 video in .mp4 format. Each frame currently renders in a little less than a minute on my Macbook Air, so the total render time was a little under an hour to create my 2.5 second-ish video.
Summary
One of my new hobbies since coming to Spain is to generate 3D models of objects in the city using a series of photos at different angles. My latest is a plate of Gilda that I ate during Bilbao Basque fest. I’m also improving my skills in Blender, which I use to correct errors in the 3D scan, and create lighting and animations.