This post was written for Deloitte Digital Mexico, as part of an internal hackathon held in January, 2018

In January, we organized a hackathon in Deloitte Digital to explore the vast possibilities of artificial intelligence and augmented reality to tackle a business challenge.

The challenge

In today’s world, there are many new products and services launched each day. Often, we can’t keep up with new gadgets, smartphones and other cool new products that pop up in the streets. For somebody that is not on top of trends and technology, it can be hard to keep up and understand what’s going on.

Next to this, our shopping experiences tend to become more personal and customized: what fits me? What do I need? Can I afford it? How can I pay for this product? Etc. Moreover, online shopping allows us to have a personal, in-home shopping experience – all from our lazy couch. We expect direct, real-time and tailored advice, because we’re all unique with different needs.

Companies are on a constant journey to increase customer loyalty. One way to achieve this, is by focusing on what matters most: how do I feel more emotionally engaged with a brand? What are they doing to answer my desires, needs and personal goals?

A final trend we’re seeing is that people are becoming ‘app tired’. We only use around 5 apps of the 70+ we have installed and are ‘careful’ using precious mobile data for installing yet another new application (that we might not even end up using). Also, downloading these apps takes up a lot of data. A recent Deloitte survey showed that 71% of the Mexican population has a data plan of 5GB or less – making data and bandwidth a precious means, not to be spent lightly.

Meet Identibot

Identibot is – excuse my French – an Artificial Intelligence powered, Augmented Reality enabled chatbot that is here to detect and answer all your product questions. And it does so real-time and on the go.

How does it work?

We’re combining 3 disruptive technologies for an optimal user (and shopping) experience:

  1. Artificial Intelligence: Google Vision API is a strong and powerful API that is able to recognize products based on images. It detects any objects and outputs its specifics, such as brand, type of product, color and category.
  2. Augmented reality: when an object is retrieved from Google Vision, its information is projected the object. This allows for a seamless and richer user experience as relevant information can pop-up where it needs to. Think, for example, about projecting audio speakers next to a laptop, sunglasses next to a summer outfit or mountain-biking gear next to a ATB. Or in our example, showing an iPhone and Google home device next to the MacBook Pro.
  3. Chatbot: we’re using Facebook’s Messenger Chatbot API to show most frequently asked product questions, and allow users to ask anything related to the product and/or their personal context. Questions like ‘Can I afford this?’ and ‘Where can I buy this’ take into consideration the user’s location, financial situation (by connecting to a banking API) and other contextual information. Moreover, this function syncs with Facebook Messenger – so feel free to continue the conversation on another channel at any time.

The revolution of the web app

In our prototype, we still use a native application to start the Identibot service. However, there is no reason why this should ‘live’ in an app. As a matter of fact, it would be far better if we could start this service from any messaging app (Telegram, WhatsApp, Facebook Messenger, etc.), this taking away any barrier to engage with the service. No heavy download, no data waste; it’s all just 1 click away in your favorite messenger app. This will create far more reach, without giving in on its functionality: all components we’ve used are compatible to be integrated in a web based solution.

What’s next?

The possibilities are endless. We’ve created a pull-factor for users to get information about a product and are able to give them contextual, relevant information on the spot. For retailers this opens up doors for direct, up and cross-selling through a better understanding of customers and their needs.

Moreover, such a service can be used for check-out purposes in the supermarket: take a picture of your basket to register (and pay) for all products, but also to ask for product-specific questions you might have.

The chatbot and its open API structure makes it possible to connect finance, sports, location and other contextual APIs – shaping a richer customer profile to give better, intelligent advice and recommendations.


In Deloitte, we facilitate hackathons and bootcamps on a weekly basis. They help creating quick value by working with a team of designers, developers and business experts. If you’re interested, please give me a ring!