Introduction

MoEngage lets you import users, events, and auxiliary data into your workspace from files (Amazon S3, SFTP, or a manual CSV upload) and directly from your data warehouse (Snowflake, BigQuery, or Databricks). This page helps you choose the right source for your use case. Once you pick a path, its dedicated setup page walks you through the entire process.

If your system requires IP whitelisting, refer to IP Whitelisting in MoEngage before setting up a data import.

Sources and Supported Entities

Each source supports a different set of entity types. Use this matrix to confirm your source can import the data you need. File Sources

Source	Registered Users	Anonymous Users	Events	Auxiliary Data
Amazon S3	✅	✅	✅	✅
SFTP	✅	✅	✅	✅
Manual Upload (CSV)	✅	✅	✅	✅

Data Warehouses

Source	Registered Users	Anonymous Users	Events	Auxiliary Data
Snowflake	✅	✅	✅	❌
BigQuery	✅	✅	✅	❌
Databricks	✅	✅	✅	❌

Entity types:

Registered Users: Users already registered on MoEngage. Also used to update existing users in bulk.
Anonymous Users: Users who have not yet registered on MoEngage.
Events (Standard and User-Defined): Standard events such as Campaign Interaction Events, as well as your own user-defined events.
Auxiliary Data: User data ingested into MoEngage temporarily for specific business purposes. Available for file-based sources only (S3, SFTP, and Manual Upload), not for data warehouses.

Choose Your Import Path

Select the import method that matches where your data lives:

Set Up a File Import

Automate imports from files dropped in Amazon S3 or an SFTP server, on a one-time or periodic schedule.

Set Up a Data Warehouse Import

Import directly from Snowflake, BigQuery, or Databricks tables and views. Connection-driven, with one-time or periodic syncs.

Manual Uploads

Upload a CSV directly in the dashboard for users or events. One-time only, no periodic scheduling.

Auxiliary Data

Import supplementary data via S3, SFTP, or CSV, using a simple primary-key mapping model.

How Imports Work

The following mechanics are common to every import source. Source-specific setup steps, limits, and failure behavior live on each source’s setup page.

Imports Dashboard

The Data Imports dashboard lists all your imports and their key details:

Column	Description
Name	The name you gave the import while setting it up.
Type	The source of the import (S3, SFTP, CSV, Snowflake, BigQuery, or Databricks). The import type, One-Time or Periodic, is shown in brackets.
Custom Segment	If the user import saves to a segment, the segment name is displayed here.
Created at	The date and time the import was created.
Last Sync Status	The status of the most recent import run as a whole, not the status of the last file within that run (see statuses below).
Files Processed	The number of files processed to date.
Rows Processed	The total number of rows processed to date.
Actions	View Details, Edit, Duplicate, or Delete an import.

Import schedule and detail times shown on the Data Imports dashboard use your app’s configured timezone. If no app timezone is set, they display in UTC by default. Set the app timezone from Settings to see these times in your preferred timezone. This display timezone is separate from the requirement that mapped timestamp columns (such as Event time or Updated at) always be in UTC.

Import Statuses

Every import on the dashboard carries one of the following statuses:

Scheduled: The run is scheduled to execute in the future.
Pending: One or more files in the run are held back because an hourly or daily import limit is reached. The run resumes on its own when the limit frees up, so no action is needed.
Processing: The run is in progress.
Successful: Every file in the run succeeded.
Partial Success: The run finished with a mix of succeeded and failed files.
Failed: Every file in the run failed. Hover over the status to see the reason.
Archived: Someone archived the import. Archiving is a manual action rather than something MoEngage does on its own, and it cannot be undone, so an archived import cannot be restored. The run and file history of an archived import stays available.
Expired: The import passed its end date. MoEngage sets this status itself, which is what separates it from Archived.

MoEngage derives the run status from the files in that run, in the following order:

Condition	Run status
Any file is processing, or scheduled files are still queued	Processing
Any file is pending because a limit is reached or an internal retry is in progress	Pending
Every file reached a final state and all succeeded	Successful
Every file reached a final state and all failed	Failed
Every file reached a final state, with a mix of succeeded and failed	Partial Success

A run that still contains a pending file stays Pending, even when other files in the same run have already failed. The run resolves to Partial Success only after every file reaches a final state. To see how each scheduled run performed, click the kebab menu under Actions, and then click View. For more information, refer to Run History. You can export a copy of an imported file for up to 60 days.

Check Import Limits

MoEngage applies separate hourly and daily import limits to three categories: User imports, Event imports, and Aux data. The Import limit overview tells you how much of each limit you have consumed, when the limits reset, which of your imports are held back, and which of them consumed the most. These limits apply to file imports from Amazon S3, SFTP, and CSV upload. Data warehouse imports use a separate limit pool that this drawer does not cover.

Open the Import Limit Overview

You can open the Import limit overview from the Data imports page, or while you are setting up an import. To open it from the Data imports page, perform the following step:

Click Check import limit, beside Import.

To open it while you set up an import, perform the following steps:

Click + Import, and then click Users, Events, or Auxiliary data.
In the dialog box, click Check import limit.

The dialog box also shows how much of your hourly and daily limit is left before you start, so you can tell whether an import is likely to be delayed before you set it up.

Know Before You Reach a Limit

MoEngage notifies you as you approach a limit, so you can reschedule an import before it is held back. A banner appears at the top of the Data imports page when you reach 80% of a limit, warning that new imports might be delayed, and again when you reach a limit, naming the category and when imports resume. When more than one limit is reached, the banner names no resume time, because the affected limits can reset at different times; open the Import limit overview to see each one. The banner appears on every tab, so a limit reached in one category is visible while you are viewing another. When you are below 80% on every limit, MoEngage shows no banner.

Limit breached banner at the top of the Data imports page, naming the daily limit for Users imports and when imports resume

Check How Much You Have Consumed

The Import limit overview shows one card for each category, with your hourly usage above and your daily usage below. The progress bar on each card shows how much of that limit you have consumed, as an amount against the limit and as a percentage. To read the usage, perform any of the following actions:

Hover over the information icon beside Usage to view when each limit resets.
Click Refresh to update the figures.
Click the User imports, Event imports, or Aux data card to see the detail for that category.

Hourly limits reset on a rolling lookback window rather than at a fixed time, so usage ages out gradually. Daily limits reset every day at 12:00 AM UTC. The drawer does not refresh on its own, so the Last updated at timestamp shows when the figures were last read. After you select a card, the Affected imports and Top contributing imports sections below apply to that category.

Import limit overview for User imports, showing the Usage cards with the reset schedule, Affected imports and Top contributing imports

Check Which Imports Are Affected

Affected imports lists the imports held back by a limit, with a banner naming the limit that was reached and when the held-back files resume.

Affected Imports banner naming the limit that was reached and when the remaining files resume

The table lists the following:

Column	Description
Import name	The name of the affected import, linked to its details page.
Type	The source and schedule of the import, for example SFTP (Periodic).
Started at	When the import started, or is scheduled to start.
Status	Pending for an import with files held back, or Scheduled for an import that has not started.
Files processed	The number of files either currently being processed or finished processing so far. Shows `--` for an import that has not started.
Files left	The number of files still pending. Shows `--` for an import that has not started.

The table lists every import held back because a limit was reached. If the hourly limit is reached, those imports resume as soon as the limit frees up. If the daily limit is reached, the pending and scheduled imports for that day resume the next day.

Check Which Imports Consumed the Most

Top contributing imports shows which of your imports used the most of the limit, so you can find the cause before you add or reschedule an import. Top contributing imports appears after at least one import in the category completes a run. The Hourly and Daily tabs change the period the table covers:

Click Hourly for the imports that ran within the current rolling one-hour window.
Click Daily for the imports that ran since the last reset at 12:00 AM UTC.

The table lists the following:

Column	Description
Import name	The name of the import, linked to its details page.
Type	The source and schedule of the import.
Started at	When the import started.
Status	The status of the import.
Consumption	The volume the import consumed, as an absolute number and as a percentage of the total limit for that category and window.

Top contributing imports with the Hourly and Daily tabs, and the highest consuming import at the top

MoEngage sorts the table by the volume each import consumed and shows the top 5. Each percentage is that import’s share of the limit for the category and window.

Trigger Imports

You can trigger imports in the following ways:

Manually: Trigger a periodic import from the imports page by clicking Actions. Triggering an import within five minutes of its scheduled time may cause an error.
API: Trigger imports using the File Imports Trigger API. Data Warehouses do not currently support these APIs.

Mapping Files and Data Mapping

A mapping file contains the mappings between each source column and a MoEngage attribute, along with the data type of the column. The file must be in JSON format. Instead of mapping columns one by one on the dashboard, you can upload a mapping file to automate the mapping.

CSV Files
JSON Files

{
  "mapping": [
    { "column": "ID", "moe_attr": "uid", "type": "string", "is_skipped": false },
    { "column": "First Name", "moe_attr": "u_fn", "type": "string", "is_skipped": false },
    { "column": "First Seen", "moe_attr": "cr_t", "type": "datetime", "datetime_format": "YYYY-MM-DD hh:mm:ss", "is_skipped": false },
    { "column": "LTV", "moe_attr": "t_rev", "type": "double", "is_skipped": false },
    { "column": "Install Status", "moe_attr": "installed", "type": "bool", "is_skipped": false }
  ]
}

{
  "mapping": [
    { "column": "customer_id", "moe_attr": "uid", "type": "string", "is_skipped": false },
    { "column": "user.first_name", "moe_attr": "u_fn", "type": "string", "is_skipped": false },
    { "column": "user.first_seen", "moe_attr": "cr_t", "type": "datetime", "datetime_format": "YYYY-MM-DD hh:mm:ss", "is_skipped": false },
    { "column": "attribution.lifetime_value", "moe_attr": "t_rev", "type": "double", "is_skipped": false },
    { "column": "attribution.device_installed", "moe_attr": "installed", "type": "bool", "is_skipped": false }
  ]
}

For each column, provide the following fields:

column (required): The column name from the source file. For Level 2 keys in a JSON file, use dot notation (key1.key2).
moe_attr (required): The MoEngage attribute to map the column to. Ensure each column maps to a unique moe_attr.
type (optional): The data type of the column. See the supported types below.
datetime_format (optional): The date-time format. Mandatory for DateTime fields only.
is_skipped (optional): A boolean field. Any column marked true is skipped during import.

Standard User Attributes for Reference

Map your source columns to MoEngage standard user attributes using the keys below. For the exhaustive list, refer to your Data Management dashboard.

Key	Attribute Name on Dashboard	Datatype	Description
`uid`	ID	String	Unique ID that the app has set for a user.
`u_n`	Name	String	Full name of the user.
`u_fn`	First Name	String	First name of the user.
`u_ln`	Last Name	String	Last name of the user.
`u_em`	Email (Standard)	String	The email address of the user. For example, `[email protected]`.
`u_gd`	Gender	String	Gender of the user.
`u_bd`	Birthday	DateTime	Birth date of the user. Use this standard attribute instead of age; data sent as age is tracked as a custom attribute.
`u_mb`	Mobile Number (Standard)	String	Mobile number of the user. For example, `918888444411`.
`moe_geo_location`	Location	Array of `[lat,lng]`	The location of the user, in the format `{"lat": 12.11, "lon": 123.122}`.
`source`	Publisher Name	String	The publisher name of the install. For example, `Google Ads`.
`revenue`	LTV	Numeric	Lifetime value of the user.
`moe_unsubscribe`	Unsubscribe	Boolean	Email unsubscribe attribute. Emails are not sent to the user when set to `true`.
`moe_hard_bounce`	Hard Bounce	Boolean	Email hard bounce attribute. Emails are not sent to the user when set to `true`.
`moe_spam`	Spam	Boolean	Email spam attribute. Emails are not sent to the user when set to `true`.

Track standard string attributes with the correct data type. For example, if First Name (u_fn) is ingested as a number or an array instead of a string, the sample users on the Create segment page fail to load with a 500 error (“There seems to be an error”). To fix this, pin the attribute’s data type to String on the Data Management dashboard and re-send the corrected data for the affected users.

Supported Attribute Types

Type	Description	Value in Mapping File
String	Any string value. For example, `ABC`, `ABC XYZ`, `ABC123`.	`"type": "string"`
Double	Any decimal value. For example, `3.14159`, `241.23`, `-123.1`.	`"type": "double"`
Boolean	For example, `true`, `false`.	`"type": "bool"`
Date Time	Any date-time value. For example, `2019/02/22 17:54:14.933`.	`"type": "datetime"`

MoEngage does not support the | (pipe) character in non-array type columns. Ensure your String, Numeric, and Boolean columns do not contain this character.

Reserved Keywords for User Attributes

MoEngage reserves the following keys. Do not use them when you map or track user attributes.

USER_ATTRIBUTE_UNIQUE_ID
USER_ATTRIBUTE_USER_EMAIL
USER_ATTRIBUTE_USER_MOBILE
USER_ATTRIBUTE_USER_NAME
USER_ATTRIBUTE_USER_GENDER
USER_ATTRIBUTE_USER_FIRST_NAME
USER_ATTRIBUTE_USER_LAST_NAME
USER_ATTRIBUTE_USER_BDAY
USER_ATTRIBUTE_NOTIFICATION_PREF
USER_ATTRIBUTE_OLD_ID
MOE_TIME_FORMAT
MOE_TIME_TIMEZONE
USER_ATTRIBUTE_DND_START_TIME
USER_ATTRIBUTE_DND_END_TIME
MOE_GAID
INSTALL
UPDATE
MOE_ISLAT
status
user_id
source

Supported Datetime Formats

Use these formats in the datetime_format field of your mapping file, or when configuring date-time columns during setup.

Datetime Format	Examples
`"datetime_format": "YYYY-MM-DD"`	2022-01-22
`"datetime_format": "YYYY/MM/DD"`	2022/01/22
`"datetime_format": "DD/MM/YYYY"`	22/01/2022
`"datetime_format": "DD-MM-YYYY"`	22-01-2022
`"datetime_format": "DD-MM-YYYY hh:mm:ss"`	31-12-2022 12:10:33
`"datetime_format": "DD/MM/YYYY hh:mm:ss"`	31/12/2022 12:10:33
`"datetime_format": "YYYY-MM-DD hh:mm:ss"`	2019-02-22 17:54:14
`"datetime_format": "YYYY/MM/DD hh:mm:ss"`	2019/02/22 17:54:14
`"datetime_format": "DD-MM-YYYYThh:mm:ss.s"`	31-12-2022T12:10:33.882
`"datetime_format": "DD/MM/YYYYThh:mm:ss.s"`	31/12/2022T12:10:33.882
`"datetime_format": "DD-MM-YYYYThh:mm:ssTZD"`	31-12-2022T12:10:33Z 31-12-2022T12:10:33+08:00 31-12-2022T12:10:33-08:00
`"datetime_format": "DD/MM/YYYYThh:mm:ssTZD"`	31/12/2022T12:10:33Z 31/12/2022T12:10:33+08:00 31/12/2022T12:10:33-08:00
`"datetime_format": "YYYY-MM-DD hh:mm:ss.s"`	2019-02-22 17:54:14.933
`"datetime_format": "YYYY/MM/DD hh:mm:ss.s"`	2019/02/22 17:54:14.933
`"datetime_format": "YYYY-MM-DDThh:mm:ssTZD"`	2019-11-14T00:01:02Z 2019-11-14T00:01:02+08:00 2019-11-14T00:01:02-08:00
`"datetime_format": "YYYY/MM/DDThh:mm:ssTZD"`	2019/11/14T00:01:02Z 2019/11/14T00:01:02+08:00 2019/11/14T00:01:02-08:00
`"datetime_format": "YYYY-MM-DDThh:mm:ss.sTZD"`	2019-02-22T17:54:14.957Z 2019-02-22T17:54:14.957299-08:00 2019-02-22T17:54:14.957299+08:00
`"datetime_format": "YYYY/MM/DDThh:mm:ss.sTZD"`	2019/02/22T17:54:14.957Z 2019/02/22T17:54:14.957299-08:00 2019/02/22T17:54:14.957299+08:00

Frequently Asked Questions

My imports have failed. How do I check what went wrong?

Click the ellipsis on the right and click View to look up the Import details. Hover over the Failed Status to learn the reason.

What if a scheduled import adds the data into a recently archived segment?

In such cases, the new data will still be added to the archived segment. You can unarchive the segment as required.

Can an import be stopped while it's running?

Once an import process starts, it can’t be stopped midway. This is because the data goes through several steps, and interrupting it could lead to incomplete or inconsistent results. It’s best to let the current import finish.

How can I stop future runs of a scheduled import?

Yes, you can stop future scheduled imports from running automatically. To do this, find the import schedule and select the Archive option from the Actions menu on the Data Imports dashboard. This will prevent it from running on its next scheduled time.

What should I do if an import seems stuck or is taking a long time?

If an import appears to be stuck or is taking longer than usual, it’s best to wait. The system has checks in place to handle these situations automatically and retry if necessary. Manually starting the same import again while it’s still processing can cause conflicts and may prevent the original import from completing successfully.

Guide

Overview - Imports