Streams - MoEngage

With MoEngage Streams, you can configure your API endpoint, define the events you want to forward, and then view a stream of data flowing into your system. You can then utilize the data to enrich your data warehouses and recommendation systems.

Use Cases for MoEngage Streams

The following are a few popular use cases for Streams:

Enrich the central data warehouse with MoEngage events such as Notification Clicked Android, Email Clicked, SMS Clicked, and so on.
Forward the Conversion Events, such as Purchase, Song Played, Bill Payment Done, and so on, from MoEngage to the external Analytics tool.
Feed campaign interaction events from MoEngage to your recommendation system.
Export the notification interaction data from MoEngage to your machine learning system for optimizing the performance of your algorithms.
Send all user events present in MoEngage to an external data lake.

Enable Streams for Your Account

Contact your Customer Success Manager to get this enabled for your account. Note that Streams is available as a part of the Streams add-on.

Streams can export data to your API endpoint or one of the partners integrated with MoEngage.

Set Up Streams for Exporting Data to Your API Endpoint

To export data from MoEngage to your servers, contact your MoEngage Customer Success Manager (CSM) or the Support team with the following information:

API endpoint to send the data. It can be something like https://api.example.com/events.
List of events that need to be sent to this API endpoint. A list of campaign events available for export is mentioned in the next section.
List of user attributes and device attributes that need to be sent to the API endpoint with each event.
If you need all campaign interaction events, we will export such events as mentioned in the next section.

You can use the following template to raise a Streams enablement request:

Support Ticket

Hey!

Please enable Streams for my Workspace.

Workspace Region: DC-01/DC-02/DC-03/DC-04
Workspace Name: MyWorkspace
Workspace ID: <my-workspace-id>

API Endpoint: [https://www.example.com/api?url_param1=vaue1](https://www.example.com/api?url_param1=vaue1)

Headers:
header1: value1
header2: value2

Authentication:
Method: No auth/Basic Auth
[Only for basic auth] Basic Auth username: "Username"
[Only for basic auth] Basic Auth password: "Password"

Events to export: All/List of events
[Optional] List of events: List of events to export

User properties to export: All/List of user properties
[Optional] List of user properties: List of user properties to export

Whitelist IPsIf your endpoints are in a Virtual Private Cloud (VPC) or not accessible publicly, you must whitelist these MoEngage IPs depending on the region of your Workspace.

Set Up Streams for Exporting Data to Partners

With Streams, you can export data directly to integrated partners so that you can enrich your marketing/analytics activities on these platforms. As of now, Streams can export data to the following partners:

Please follow partner-specific integration documents for setting this up.

Campaign Interaction Events

The following is a list of Campaign Interaction Events that MoEngage generates, which you will often need to be sent to your API endpoint:

Event Name	Event Code	Channel
Email Sent	`MOE_EMAIL_SENT`	Email
Email Deferred	`MOE_EMAIL_DEFERRED`	Email
Email Delivered	`MOE_EMAIL_DELIVERED`	Email
Email Dropped	`MOE_EMAIL_DROP`	Email
Email Bounced	`MOE_EMAIL_HARD_BOUNCE`	Email
Email Soft Bounced	`MOE_EMAIL_SOFT_BOUNCE`	Email
Email Opened	`MOE_EMAIL_OPEN`	Email
Email Clicked	`MOE_EMAIL_CLICK`	Email
Email Unsubscribed	`MOE_EMAIL_UNSUBSCRIBE`	Email
Email Spam Complained	`MOE_EMAIL_SPAM`	Email
SMS Sent	`SMS_SENT`	SMS
SMS Delivered	`SMS_DELIVERED`	SMS
Notification Received Android	`NOTIFICATION_RECEIVED_MOE`	Push
Notification Clicked Android	`NOTIFICATION_CLICKED_MOE`	Push
Notification Swiped Android	`NOTIFICATION_CLEARED_MOE`	Push
Notification Sent iOS	`n_i_s`	Push
Notification Clicked iOS	`NOTIFICATION_CLICKED_IOS_MOE`	Push
Notification Received Web	`NOTIFICATION_RECEIVED_WEB_MOE`	Push
Notification Clicked Web	`NOTIFICATION_CLICKED_WEB_MOE`	Push
Connector Sent	`MOE_CONNECTOR_SENT`	Connector
Card Sent	`MOE_CARD_SENT`	Cards
Card Delivered	`MOE_CARD_DELIVERED`	Cards
Card Viewed	`MOE_CARD_VIEWED`	Cards
Card Clicked	`MOE_CARD_CLICKED`	Cards
Mobile In-App Shown	`MOE_IN_APP_SHOWN`	Mobile In-Apps
Mobile In-App Clicked	`MOE_IN_APP_CLICKED`	Mobile In-Apps
Mobile In-App Closed	`MOE_IN_APP_DISMISSED`	Mobile In-Apps
On-site Message Shown	`MOE_ONSITE_MESSAGE_SHOWN`	On-site Messaging
On-site Message Clicked	`MOE_ONSITE_MESSAGE_CLICKED`	On-site Messaging
On-site Message Closed	`MOE_ONSITE_MESSAGE_DISMISSED`	On-site Messaging
User Entered Flow	`USER_ENTERED_FLOW`	Flows
User Exited Flow	`USER_EXITED_FLOW`	Flows
User Added to Control Group	`MOE_CAMPAIGN_CONTROL_GROUP`	All channels

With the events above, the following event attributes will be exported by default: campaign_id, campaign_name, campaign_type, campaign_channel.
The following event attributes are not supported for export in streams currently: email_subject, email_click_url, inapp_widget_clicked, onsite_message_url_clicked.

For more information on when the campaign events and attributes are tracked, refer to Data Exports Glossary.

Authentication Methods

Streams supports the following types of authentications:

No auth
Basic Auth: Streams supports the standard Basic Auth implementation. You must provide us with the username and password while configuring Streams. All data related to authentication is always encrypted in our systems.

To support other authentication methods such as API Key, Bearer Token, and so on, you can communicate the same through the enablement ticket and provide us the Header key and static value to be passed. We will not be able to support dynamically refreshing API keys and tokens.

API Request Format

When Streams sends an event to your API endpoint, the request format will be as below: Headers: "Content-Type":"application/json" Request Body:

JSON

{
    "app_name": "App Name",
    "source": "MOENGAGE",
    "moe_request_id": "moengage unique request id for each request",
    "events": [{
        "event_name": "Notification Received Android",
        "event_code": "NOTIFICATION_RECEIVED_MOE",
        "event_uuid": "moengage unique id for each event",
        "event_time": 1580967474,
        "event_type": "CAMPAIGN_EVENT",
        "event_source": "MOENGAGE",
        "push_id": "recipient device’s push token",
        "uid": "<MoEngage customer_id>",
        "event_attributes": {
            "campaign_id": "353df897hkbh67658",
            "campaign_name": "Name of the Campaign",
            "campaign_type": "Event Trigger",
            "campaign_channel": "Push"
        },
        "user_attributes": {
            "moengage_user_id": "moe_internal_user_id",
            "user_attr_1": "user_attr_val1",
            "user_attr_2": "user_attr_val2"
        },
        "device_attributes": {
            "moengage_device_id": "moe_internal_device_id",
            "device_attr_1": "device_attr_val1",
            "device_attr_2": "device_attr_val2"
        }
    }]
}

Streams Data Glossary

For the full list of updated events and attributes, refer to Data Exports Glossary.

The keys in the API request and their description are:

Key	Description
workspace name	Your workspace name in MoEngage.
source	`MoEngage` to identify the requests coming from MoEngage.
event_name	Display Name of the event as seen on the MoEngage dashboard.
event_code	Raw event code as present in the MoEngage system.
event_uuid	Unique event identifier for de-duplication.
event_time	Time of event in UTC (epoch time in seconds).
event_type	`CAMPAIGN_EVENT` or `USER_ACTION_EVENT`.
event_source	Source identifier (value = `MoEngage`).
push_id	Push token (available for Push-related events).
email_id	Recipient email (available for Email-related events).
mobile_number	Recipient mobile number (available for SMS-related events).
uid	MoEngage `customer_id` unique identifier.
campaign_name	Campaign name in MoEngage.
campaign_id	Campaign ID in MoEngage.
event_attributes	Dictionary of additional event attributes.
campaign_channel	Channel type: Push / Email / SMS.
user_attributes	Dictionary of additional user properties.
moengage_user_id	MoEngage internal user ID.
device_attributes	Dictionary of additional device attributes.
campaign_type	Type: Periodic, Active, One-time.
variation_id	Present for campaign variations.
locale_id	Present for campaigns with multiple locales.
locale_name	Display name of the locale.
parent_campaign_id	Identifier for localized campaign parents.
parent_flow_id	Identifier for campaigns within a MoEngage Flow.
parent_flow_name	Name of the MoEngage Flow.

Limitations

Please be aware of the following constraints or limitations while integrating Streams with third-party apps.

We can send data to only one static endpoint where the access token or link has no expiry. We cannot integrate it on a dynamic endpoint where the access token is refreshed every few hours.
We cannot modify the body or key names while streaming the data from MoEngage.

Sync Events from One MoEngage App to Another

With MoEngage Streams, it is possible to sync your events from one MoEngage App to another. The following are some use cases:

Cross-Product Promotions: If you have teams across various product offerings, keeping two apps synced can help you cross-promote your products to upsell and increase conversion.
Centralized Analysis: Businesses with multiple apps can do cross-app analysis to unlock higher LTV, provide opportunities for cross-selling, and increase brand loyalty.

Stream Events to Another MoEngage App

Get your Streams endpoint

Speak to your CSM to get your dedicated Streams endpoint for your destination app.

Set up Streams on source App

Follow the enablement steps to configure the source App with the following details:

Endpoint: Use the dedicated MoEngage endpoint provided for your destination app.
Authentication: Enable Basic Auth. Use your destination app’s Workspace ID as Username and its Data API Key as Password.
Header: Provide "Content-Type" : "application/json".
Events/Properties: Select the list of events and user properties you want to stream.

Frequently Asked Questions

What is the throughput, average volume, and batch size?

The throughput and average volume of each API request depends on the volume of selected events captured into MoEngage. The default batch size (number of events for each API request) is 100.

What is the retry mechanism of Streams?

In case a batch of events fail (any non-2XX response), the entire batch is retried. In total, three retry attempts are made with intervals of 30, 60, and 120 seconds.

If I pause Streams, can I still get the data later?

No. If you pause Streams, data is not collected for exports and cannot be replayed at a later date.

Does Streams guarantee the latest user attributes?

No. Streams is built for near real-time event export. Since user attributes are updated asynchronously, we cannot guarantee the latest values in the export.

Can I export historical data using Streams?

No. You can only export data generated after the Streams configuration is enabled.

​Use Cases for MoEngage Streams

​Enable Streams for Your Account

​Set Up Streams for Exporting Data to Your API Endpoint

​Set Up Streams for Exporting Data to Partners

​Campaign Interaction Events

​Authentication Methods

​API Request Format

​Streams Data Glossary

​Limitations

​Sync Events from One MoEngage App to Another

​Stream Events to Another MoEngage App

​Frequently Asked Questions

Use Cases for MoEngage Streams

Enable Streams for Your Account

Set Up Streams for Exporting Data to Your API Endpoint

Set Up Streams for Exporting Data to Partners

Campaign Interaction Events

Authentication Methods

API Request Format

Streams Data Glossary

Limitations

Sync Events from One MoEngage App to Another

Stream Events to Another MoEngage App

Frequently Asked Questions