Gamified Content in Augmented Reality (AR)

Introduction

Jana, the most famous Croatian bottled water, has launched a new label for its bottles. With the creative idea of the McCann Zagreb agency and our production, it came to life in AR (augmented reality). Jana has returned to its original label, featuring a painting by Ivan Rabuzin, a Croatian naïve artist, which has now been given new life with the help of artificial intelligence (AI). The intervention of artificial intelligence has given a modern look to the old painting, and we further brought the newly created landscape to life with the help of AR technology. Each Jana bottle has a QR code that, when scanned, animates the label and builds a wondrous world of naïve art in augmented reality around the bottle. Jana has revived the old label, and we have brought it into the additional dimension of augmented reality. Read more about how we brought the label into the 3D world below.

Development of visual styles – Premium

The first step was the interpretation of that visual into a detailed 3D model with textures for the needs of billboard advertising and digital screens. It was necessary to develop a high-poly 3D model of nature that we can control with quality. That’s why we developed a kind of organic approach to Rabuzin’s pointillism, as one of his supporting visual elements. In order for things to look natural, it is necessary to add certain irregularities to that mathematical artificial structure.

We tried many variants until finally we captured the best ratio of point structure and organic development. We were aiming for a kind of magical realism where things that don’t exist seem extremely real. The first part of the story is rounded off with a wearable visual of the bottle in nature.

Visual style optimization for AR

But the real task is still ahead of us, for the needs of augmented reality it was necessary to develop a new visual style because highly detailed models cannot be rotated on mobile phones. The idea had to be transferred into a low-poly 3D model that would be suitable for augmented reality (AR), while still keeping the game of “old and new”. It was necessary to make a kind of “Rabuzination of visuals”, to simplify and stylize the photo-realistic presentation.

Augmented reality systems work best when used through apps on cell phones. Then the full potential of newer generation mobile phones that have multiple cameras or high-precision LiDAR cameras can be used. LiDAR cameras use laser beams to measure distances to objects and create detailed 3D models of the environment. But users don’t want to install another app. In any case, they don’t want to install an app just to try it out.

Web AR, augmented reality through a web browser

This is where the Web AR solution comes into play, which enables access to augmented reality content through an internet browser, without the need to install an application. Such a system has many limitations because it tries to adapt the content to a large number of devices, new and old, that access it. That’s why some advanced technologies such as LiDAR cameras (recognizes depth in space) or SLAM technology, which allows devices to simultaneously map an unknown environment and determine their position within that environment, work less well through Web AR than through the application. SLAM (Simultaneous Localization and Mapping) enables devices to simultaneously map space and determine their position within that space. However, Web AR has an indisputable advantage over applications: the user gets what he wants most – simple content consumption!

How does augmented reality work?

The first step of this kind of experience is usually scanning the QR code. Jana’s label has a QR code on it, which is appropriately designed in the form of a drop of water. By scanning that QR code, augmented reality is triggered, that is, the label from the bottle “comes to life” and spreads its poetic Rabuzin landscape all around the user.

In order for the impression of augmented reality to be complete, the graphic elements that we place in the 3D space must be moved together with the space. They must look like they belong in that space. We can do this “moving” in several ways:

User experience – UX

In the specific case of Jane, we combined two technologies in order to get the best experience for users, which is the most important thing, after all. First, we look for a picture marker, i.e. we scan the label on the bottle with the cell phone camera. Once the label is recognized, SLAM technology is launched, which creates a whole new world from one point.

Gamification of content

As from the very beginning of the project there was also talk about a game that would be an integral part of the user experience, we designed a simple AR game in which you need to find and collect 5 discarded bottles in order to clean nature.

Although it is a simple game, attractive visuals and cute animations attract attention, which drastically increases the chance that the message from Jana’s world will be noticed. Users are more engaged when they play, and because of this gamification of content, it is easier to reach them. It is only important that the messages and mechanics of the game are harmonious in that story.

There is an old saying that the world rests on the young, and as new technologies are closer to the younger generations who grew up with cell phones and tablets, gamified experiences become one of the best forms of “communication”.

P.S. See our services Creative Solutions, Multimedia Installation, Application and Game Development