It’s 2029, mid November. You are feeling the urge to do something creative for Halloween and time is running out. You pick up a pumpkin at Trader Joe’s, grab some spiced rum, and head home to cook up some ideas.
With cocktail in hand, you hollow out your pumpkin. Setting it aside on the table, you toss on your inconspicuous lightweight AR glasses. You ask your DA (digital assistant…no attorney necessary) to load up sports related templates. Within seconds, you are able to swipe through and see each perfectly overlaid and fitted to your pumpkin.
Ah yes, this is the one, you proclaim! Fresh off his Superbowl MVP, you decide to immortalize Justin Jefferson and his classic Griddy celebration. You select the template, grab your carving tools, and put in light work.
Welcome back to 2022 – Let’s be honest, most of us are not professional pumpkin carving artists. We enjoy picking out what to carve and initially the fun and mess of doing it, but we don’t put much optimism in the final outcome.
Also, if we decide on a pattern that is more than the standard triangle eyes, triangle nose, and Charlie Brown shirt for a mouth, we are probably gonna use paper templates. That means aligning a crinkly page to the round, seemingly always wet pumpkin. If the tape holds, we administer a session of acupuncture and poke the pumpkin a thousand times, hoping to transfer the design to the skin itself.
If that goes according to plan, we are left staring at a matrix of dots. Um…and that should be enough to guide your carving adventure? 🤷♀️
If you’ve been down that road, you know the challenges. It’s messy and making sense of your polka dot stencil work is hard on the brain. Results are anything but guaranteed. Let’s see if AR can change that.
For this project, I am using Microsoft HoloLens 2 AR glasses. I created the custom software I use for these types of uses.
I’ve done prior non-pumpkin projects using AR glasses to overlay a digital template with great success. The roundness of a pumpkin, however, is a new wrinkle. I am confident it will either work very well… or be a total disaster; no in between.
I have some rough ideas for what to carve. I’m leaning towards a sports theme because my Philadelphia Phillies have just reached the World Series and my Minnesota Vikings are off to a fantastic start.
I could carve the logo of the teams, but that seems…kinda boring, no? Instead, I want to capture an energetic and emotional moment in time. Googling the teams and looking for iconic images, one quickly stands out – my man Justin Jefferson has been setting the NFL on fire, breaking records and laws of physics. More importantly, he’s responsible for kids and adults all over the world mimicking his signature dance, the Griddy. We have a winner!
--- griddy gif
With the pattern decided, I need to get it to a usable form. I can overlay the image as is, but it will demand skillful artistic techniques when carving. I don’t possess those, so I need to simplify the image.
I bring the image into Photoshop. I remove unneeded details and reduce the picture to just outlines. I make those outlines a bold red color. Experience has taught me that bright red lines are the easiest to see when overlaying digital templates.
The scene seems a little narrow and the pumpkin face has a lot of real estate. I want to add more width to the design, so I add the Vikings stylized name as the background. I like it!
With the final design set, I break it down into several layers. Separating layers will allow me to show them individually through the AR glasses. I plan on using this as a sort of sequencing, so I carve things in the right order.
I’ll spare us the deep technical details of the software side. High level – I threw together a Unity app that will allow me to move and scale a sphere. This sphere is wrapped in the design layers I created in Photoshop. Once I get started, my hands will be occupied with carving tools and pumpkin slime, so hand gestures are out the window to change layers. I added functionality to allow me to toggle layers using voice commands instead.
One more step before the carving commences. I set up a little station and clamp my pumpkin in place. I don’t want this thing moving around on me or my digital overlay.
With everything in place, I take that first step towards creating my masterpiece. I use my X-acto Knife to trace the first layer. This helps create a nice clean edge for peeling away pumpkin skin in the next step.
I can’t actually see the incisions made on the pumpkin with my knife. This would be a problem if I didn’t have the magic of AR and a perfectly visible holographic template.
Setting the carving tool (actually, I think it’s a sculpting tool 🤔🤷🏽♀️) edge on a virtual red line, I make a firm half scoop, half scrap motion. I repeat this over and over, slowly removing little peelings of the skin. The AR does a great job at letting me see which side of the red outline I should be removing. This would be an unfun brain challenge using traditional methods.
The first layer is going easier than expected. There is a bit of a weird AR glasses phenomenon going on that is playing into it. In some ways, seeing holograms and digital overlays is a skill you can improve. Without getting into the heavy stuff, today’s AR can be hard to tell exactly where an object is in space. Think of this, when you look straight down at a coffee cup on a table, you know the bottom of the cup is resting on the top of the table. Holograms don’t respect physics. So when you put a holographic coffee mug on a real table, it may be slightly above the table top, or slightly IN the table top. It’s a bit trippy to see and plays subtle tricks with your mind, especially close up. In time the technology will improve and your glasses will put fake objects precisely on top of real objects, but in the meantime, relying on experience helps.
One other challenge, the imperfect pumpkin doesn’t match the sphere exactly. This is playing small tricks of perspective with my eyes when trying to line up the tool edge with the pumpkin surface and digital template. Having that “trained eye” is helping me reduce the negative effects.
Now I’m in the groove. I’ve gotten the hang of the tools and am working through the layers nicely. My technique is getting a little more refined, allowing me sharper details than I could muster at the beginning. Every so often, I pop my DeWalt flashlight into the pumpkin to check progress. Each time, I’m inspired to carry on.
I don’t know exactly how long I’ve been at it, but my neck is telling me it’s been a couple of hours now. Over the years of building enterprise AR solutions, I’ve developed a rule of never putting users in uncomfortable positions. As we are our own worst enemy, in the pursuit of carving a perfect pumpkin, I threw caution to the wind. This project has me staring straight down for extended periods of time without relief – a no-no when wearing a 1.25lb HoloLens on your dome. Yep, I feel this one… but the carving must go on!
The final layer has me scraping away the background behind the Vikings lettering. There is no mystery in how this is going to turn out now. All of the guesses, assumptions, unforeseen challenges, have been peeled away. What remains is a testament to the beauty of fusing art and technology.
The grand reveal hits me even harder. With flashlight inserted, I set the pumpkin upright. Turning off the overhead lights, I spin around and see my efforts in full. I can’t help but let loose a string of expletives. My eyes are illuminated with a scene worthy of the most viral touchdown dance since the Ickey Shuffle. I cannot believe how well this turned out.
This exceeded my every expectation. The detail, the accuracy, even attempting the ambitious scene itself – AR made this possible. It is a pure example of how AR takes your existing skills and abilities and supercharges them. In a way, it augments your capabilities, opening all kinds of new doors.
Beyond pumpkins, there are endless ways to use AR for templating – installing floor tiles accurate to a desired pattern, assisting in mural paintings, wrapping cars with unique custom designs, to name a few.
As cost, size, and comfort of glasses significantly improve, more uses will become feasible. In time, AR will become a mainstay in our everyday lives.
In these early days, opportunities are abundant. If anything about this project piqued your interest, reach out. I know you have ideas and thoughts and I’d love to hear them or work with you on them.
Cheers,
Anthony
In the years to come, AR glasses will have a profound effect on how we work, play, and get things done. Let’s not talk hypotheticals. Let’s craft the vision, roll up our sleeves, and build that reality!