Engagement Analytics – What, How and When to Measure [#GROWTH19 Wrap-up]

Reading Time: 7 minutes

Bonus Content The Modern Marketer’s Quick Guide to Email Marketing [Download Ebook] Beginner’s Guide to Omnichannel Marketing for 2021 [Download Ebook] Customer Lifecycle Marketing Campaigns: An In-depth Guide for 2021 [Download Ebook]

At MoEngage, we believe what cannot be measured cannot be managed. There’s a humungous amount of data and a large number of tools available for analytics, which can get overwhelming for marketers and product owners sometimes.

We sat down for a panel discussion with Ms. Pooja Ravishankar, Category Marketing Head of Big Basket, Mr. Abhishek Joshi, CMO of MX Player, and Mr. Nitin Sethi, VP Digital of Indigo Airlines to understand how they use analytics to measure and judge user engagement.

The panel discussion was moderated by Mr. Gaurav Chhaparwal, Analytics Leader and Advisor of MoEngage. We bring you the excerpts from the discussion.

In analytics, we say what gets measured gets done. What’s that one metric that best represents your user engagement, and why do you think it is the best metric?

Nitin: For us, the most important metric is of people who have flown with us in the last three months. We analyze how many of them are flying with us again, i.e., is it their second or third trips within 3-6 months. It helps us to find ways to retain the customer. It adds to the top line, bottom line, and user retention. This is the best parameter for any transactional service-driven platform.

Pooja: In grocery, everyone requires fruits and vegetables at least once a week. That is the frequency of purchase we look at in Big Basket. Another important metric is how many times has a customer come back every calendar month of a year. It helps us to determine that the customer has not fallen off the bridge and is still active on the app. We analyze how much is a customer purchasing from us in a month, how many times is he coming back in a year and how diverse is the number of categories across the lifetime, i.e., does the person who typically orders rice and lentils, also purchase fruits and vegetables or other FMCG products on the app. We have found that our loyal customers are the ones who buy from a wide variety of categories.

Abhishek: Two metrics make a lot of sense in my business. One is returning users, i.e., how many users are coming back to my platform. MX Player was launched recently, but it has been in existence for almost five years now. It is one of the unique platforms which has seen a transition in offerings because we kept adding new offerings to the platforms. The metric of returning users is important because it tells me how the platform, the UI/UX, and the content is performing. The second metric we measure is the time spent by the user on a piece of content as it adds to the revenue, engagement, and loyalty.

How do you act upon the data you receive? Can you give us an example of how you make decisions based on specific data points?

Nitin: Our business has four to five core segments. There are millennials; there are first time travelers, business travelers who take flight in the morning and return in the evening, family travelers and leisure travelers. First, we understand these segments well. For example, we know somebody who is on a short trip or traveling alone will be fine to sit anywhere on the plane. They usually do not have any specific seat preferences and may not buy a meal. On the other hand, a passenger traveling with wife and kids would be willing to pay extra for a preferred seat and meals. Once you get these insights, you will find a pattern that will help you to decide what to offer to each segment. You need to personalize the solution for all types of audiences who interact with you.

Pooja: The audience knows that there are multiple applications of data. However, I will give you an example of a tool that we built, which is analytical in the application. Typically, grocery buying works in two modes. There is one set of products that you repeatedly buy like vegetables, fruits, and staples. The other set is where you keep discovering as newer products get launched in the market. For the first set, we have built something called ‘the smart basket,’ which runs on an algorithm that predicts what you are running out of in your kitchen. We keep moving up the item in the list. The smart basket has been in existence for the last five years. We are constantly making it smarter by adding new elements to it. For example, we notice a shift in consumer’s buying with most of them shifting from regular products to organic products. So, we have both organic potato and regular potato. Once the customer purchases the organic potato, we start moving the regular potato down the list to upsell the organic ones to the customers or with the mango season around the corner; we will start adding it to the list. These are a few smart things that we do to personalize the offerings of the app and to improve the product and convenience for the customer. However, we also face challenges because there cannot be a uniform solution to everything. For example, we have to be careful not to offer non-vegetarian recommendations to vegetarians while making basket recommendations.

Abhishek: MX players may have launched recently, but we already have 75 million Daily Active Users (DAU) and 175 million Monthly Active Users (MAU) because we have been there for five years. The journey between data to information to insights does not happen overnight. You need to spend time on it. Once your insights come, you have to know how to make a strategy for programming and content, user acquisition, etc. That is why we waited for a year or a year and a half to announce our official launch. In three weeks, we were declared the no.2 OTT platform in the country. We are not a platform that showcases content aggregated from the outside world; we make our own content. However, one must remember that the content can’t come from thin air because even if the content is based on creativity, it is also backed by science that includes determining who will see it, which geography to target, and so on. It is a continuous 24-hour job, wherein we collect close to 400 billion data points a day. We do not analyze every data point, but we pull out the relevant ones to see how it works.  

How do you act upon the insights once you receive them?

Nitin: In our business, user acquisition happens on different channels. For example, a fair amount of bookings come from Online Travel Agency (OTA) and direct channels such as mobile apps and websites, SME and agents. Once all this information is collected, and the booking pattern is evaluated, you will find that people who come from OTA have a different way of transacting because the enablement happed from them and you don’t have the information of those users until they check their flight status or do web check-in. You have limited scope to cross-sell and upsell to them as compared to someone who directly comes to your platform and is acquired by you. In this case, you can do a meaningful cross-sell and upsell. There is a very thin line between intrusive experience and great user experience. You have to use these insights properly to not cross this line. For example, 30-40% of the bookings come from our SMEs and agents or a personal assistant. There are chances that they may or may not know what meal you would like to have because your preference may change based on morning flight or evening flight. Here, you should know when to cross-sell and up-sell without pushing it on to the user.

Pooja: There’s lots of data in the e-commerce and digital context, so it’s important to have a very data-driven culture in the organization. Everybody in the organization should learn to check data before a certain decision is taken. They should be trained to interpret data correctly because it can also be misleading at times. The second thing is to have the right technology to back this data. In this ever-evolving landscape, it is important to have the best technology that gives you the right data.

Abhishek: We keep it simple. For us, knowing and evaluating the consumer’s journey is very important. We analyze data such as why has a consumer come to the app and what has he clicked on. Has he clicked on the content or has he clicked on the thumbnail because of the content he saw? We further look at what is consumed, did the user exit or go to another piece of content in the app. That is where cross-selling comes in – I should be able to sell him content similar to the one he clicked by using machine learning, PI tool, and recommendation engine. I have also come to an understanding that people who search for content will always consume long-form content, while people who don’t search for content, will consume the short-form ones. 60% of the conversion happens on the short-form content. Users consume different types of content at different time. For example, the user may be interested in watching the news when he wakes up. In the afternoon, the same person may be interested in watching short form, humor content, at late night, it could be something else. Hence, it’s important for us to show the customer content that is very close to his interest at a particular time of the day.

Thank you, Nitin, Pooja, and Abhishek for sharing your valuable insights on how you use data points to take critical business decisions.

We have come to understand that it’s essential to analyze both the width and depth of engagement to retain and engage continuously with the users. To gauge that, marketers and product owners should be able to interpret the data correctly because nothing is more dangerous than having a wrong interpretation of the data.