Introduction
MoEngage allows customers to import users and events through files stored in various sources like S3 Buckets and SFTP servers, as well as directly from tables in your data warehouses. MoEngage Dashboard enables you to configure multiple imports directly by defining file locations, configuring one or more file import sources, and import methods.Supported Sources
MoEngage currently supports importing files from: MoEngage also supports importing from data warehouses:Types of Imports
MoEngage can import the following from your sources:- Registered Users: These are users who are already registered on MoEngage. You can also use this type of import to update existing users in bulk.
- Anonymous Users: These are users who must register on MoEngage.
- Events (Standard and User Defined): MoEngage can import standard events like Campaign Interaction Events as well as your own user-defined events.
- Auxiliary Data: There are user data temporarily ingested into MoEngage for specific business purposes.
Prepare the Files
Before configuring the imports on the MoEngage dashboard, make sure that all your files are in the expected format and within the sanctioned limits.Supported File Types
| Source | S3 Imports | SFTP Imports | Upload | All Sources | |||
|---|---|---|---|---|---|---|---|
| File Type | Users | Events | Users | Events | Users | Events | Aux |
| CSV | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| JSON | ✅ | ✅ | ✅ | ✅ | ❎ | ❎ | ❎ |
Naming Conventions
<import type>_<date time format>.<file_extension>.. The final name depends on the type of import and the date-time format you pick.
Import Type
- Registered users
- File names should start with the prefix
registered_user_data_ - Example:
registered_user_data_01311997.csvorregistered_user_data_01311997.json
- File names should start with the prefix
- Anonymous users
- File names should start with the prefix
anonymous_user_data_ - Example:
anonymous_user_data_jan_2022.csvoranonymous_user_data_jan_2022.json
- File names should start with the prefix
- Events
- File names should start with the prefix
<event name>_ - In certain cases, the Event Name and Display Name can be different. To manage your events, go to the Data Management page in MoEngage.
- For example, if the event name is “Purchase Summary”:
Purchase_Summary_31011997.csvorPurchase_Summary_31011997.json - For example, if the event name is “Purchase_Transaction”:
Purchase_Transaction_19970131_1259.csvorPurchase_Transaction_19970131_1259.json
- File names should start with the prefix
File Extensions Supported
- Imports without having File Encryption enabled support only the following extensions:
.csv,.json - Imports having File Encryption enabled support only the following extensions:
.csv.asc,.json.asc,.csv.pgp,.json.pgp
Date Time Format
Your file names should end with the suffix<date time format>.<file extension>This “date-time” represents the day on which the import executes. For example, if the import is executed on “7th January 2022”, and the selected date-time format is ddmmyyyy, MoEngage looks for all the files that end with the suffix 07012022.csv or 07012022.json
MoEngage supports multiple date-time formats:
| Type | Description | Example |
|---|---|---|
{mm}_{yyyy} |
| |
{ddmmyyyy} |
| 31012022 |
{yyyymmdd} |
| 20220112 |
{mon\_yyyy} |
| jan_2022 |
{yyyymmdd}_{hhmm} |
| 20220131_1259 |
<custom name>_<date time format>.<file extension>.Filenames should contain the custom prefix name and date-time format separated by ”.” The custom name should consist of letters, numbers, a space, and ”.”For example: abc_31012024.csv or abc_31012024.jsonValid File Name Examples
The following are examples for valid file names:- Registered users import with date time format
{yyyymmdd}_{hhmm}have file names likeregistered_user_data_20220131_1259.csvorregistered_user_data_20220131_1259.json - Anonymous users Import with date time format
{mon_yyyy}have file names likeanonymous_user_data_feb_2022.csvoranonymous_user_data_feb_2022.json - Item purchased event import with date time format
{ddmmyyyy}have file names like ItemPurchased_31012022.csvorPurchased_31012022.json
File Structure
MoEngage supports importing data from CSV and JSON files. Your files must adhere to the following rules:- CSV Files
- JSON Files
- The first row of your file must be the column names. MoEngage always treat the first row of your file as the header row.
- Ensure there are no duplicate column names in your file. Do note that column names are case-sensitive.
- Ensure there are no missing column names.
- MoEngage only supports the comma (,) as a delimiter.
- Avoid empty rows (either the row is blank, or all the columns in that row are blank).
Sample Files
Here are some sample files:- Registered users import: Your files should have a column that can be mapped to the user ID (user identifier to identify User Profile in MoEngage).
- Anonymous users import: Your files should have a column (email, mobile, etc) that can be mapped to an anonymous ID. An anonymous ID such as email, mobile, etc can help you identify users who have used your app/website but have not yet signed up.
- Event Import: Your files must have separate columns for event time and user ID.
Manifest Files
A manifest file contains mappings between the source column and a MoEngage Attribute. It also contains the data type of the column. This file must be in JSON format. An example of a manifest file is shown below. Using the MoEngage Dashboard, you can map the columns in your CSV or JSON files to attributes inside MoEngage. Or optionally, you can also choose to upload a manifest file.- CSV Files
- JSON Files
- “column”: (required) Represents the column name from the source file. When mapping Level 2 columns of a JSON file, you can use dot-notations: key1.key2.
- “moe_attr”: (required) The MoEngage Attribute (attribute name) to which you want to map the source file column. You can view the number of Events, User Attributes, Volume of User and Events on the Data Management page. Ensure each column is mapped to a unique “moe_attr”.
- “type”: (optional) This field represents the data type of the column. Refer to the table below for the acceptable “type” formats for attributes. Refer to the Data Management page for more information.
- “datetime_format”: (optional) This field indicates the date and time format. It is mandatory for DateTime fields only. Refer to the table below for the complete list of supported date-time formats.
- “is_skipped”: (optional) This field represents whether MoEngage should import the column or not. It is a boolean field, and any attribute marked as true will be skipped during import.
Some other standard user attributes in MoEngage for reference:
| Key | Attribute Name on Dashboard | Datatype | Description |
|---|---|---|---|
| name | Name | String | Full name of the user. |
| first_name | First Name | String | First name of the user. |
| last_name | Last Name | String | Last name of the user. |
| Email (Standard) | String | The Email Address of the user. For example, [email protected] | |
| age | Age | Numeric | Age of the user |
| gender | Gender | String | Gender of the user |
| mobile | Mobile Number (Standard) | String | Mobile Number of the user. For example, 918888444411 |
| moe_geo_location | Location | Array of [lat,lng] in double in the format {"lat": 12.11, "lon": 123.122} | A sample value would be the location of the user. For example: {"lat": 12.11, "lon": 123.122} |
| source | Publisher Name | String | This is the Publisher Name of Install. For example, Google Ads |
| revenue | LTV | Numeric | Life Time Value of the user. |
| moe_unsubscribe | Unsubscribe | Boolean | Email Unsubscribe Attribute. Emails are not sent to the user when the set value is true. |
| moe_hard_bounce | Hard Bounce | Boolean | Email Hard Bounce Attribute. The emails are not sent to the user when the set value is true. |
| moe_spam | Spam | Boolean | Email Spam Attribute. The emails are not sent to the user when the set value is true. |
Supported User Attribute Types
| Type | Description | Value in Manifest |
|---|---|---|
| String | Any string value. Examples of acceptable values: ABC, ABC XYZ, ABC123, etc. | "type": "string" |
| Double | Any decimal value. Examples of acceptable values: 3.14159, 241.23, -123.1 | "type": "double" |
| Boolean | Examples of acceptable values: true, false | "type": "bool" |
| Date Time | Any date time value. The supported formats are covered in the next table. Example of acceptable values: 2019/02/22 17:54:14.933 | "type": "datetime" |
Supported Datetime Formats
| Datetime Format | Examples |
|---|---|
“datetime_format”: "YYYY-MM-DD" | - 2022-01-22 |
“datetime_format”: "YYYY/MM/DD" | - 2022/01/22 |
“datetime_format”: "DD/MM/YYYY" | - 22/01/2022 |
“datetime_format”: "DD-MM-YYYY" | - 22-01-2022 |
“datetime_format”: "DD-MM-YYYY hh:mm:ss" | - 31-12-2022 12:10:33 |
“datetime_format”: "DD/MM/YYYY hh:mm:ss" | - 31/12/2022 12:10:33 |
“datetime_format”: "YYYY-MM-DD hh:mm:ss" | - 2019-02-22 17:54:14 |
“datetime_format”: "YYYY/MM/DD hh:mm:ss" | - 2019/02/22 17:54:14 |
“datetime_format”: "DD-MM-YYYYThh:mm:ss.s" | - 31-12-2022T12:10:33.882 |
“datetime_format”: "DD/MM/YYYYThh:mm:ss.s" | - 31/12/2022T12:10:33.882 |
“datetime_format”: "DD-MM-YYYYThh:mm:ssTZD" | - 31-12-2022T12:10:33Z - 31-12-2022T12:10:33+08:00 - 31-12-2022T12:10:33-08:00 |
“datetime_format”: "DD/MM/YYYYThh:mm:ssTZD" | - 31/12/2022T12:10:33Z - 31/12/2022T12:10:33+08:00 - 31/12/2022T12:10:33-08:00 |
“datetime_format”: "YYYY-MM-DD hh:mm:ss.s" | - 2019-02-22 17:54:14.933 |
“datetime_format”: "YYYY/MM/DD hh:mm:ss.s" | - 2019/02/22 17:54:14.933 |
“datetime_format”: "YYYY-MM-DDThh:mm:ssTZD" | - 2019-11-14T00:01:02Z - 2019-11-14T00:01:02+08:00 - 2019-11-14T00:01:02Z-08:00 |
“datetime_format”: "YYYY/MM/DDThh:mm:ssTZD" |
|
“datetime_format”: "YYYY-MM-DDThh:mm:ss.sTZD" |
|
“datetime_format”: "YYYY/MM/DDThh:mm:ss.sTZD" |
|
“datetime_format”: "YYYY-MM-DD hh:mm:ss.sTZD" (only for Snowflake Imports) |
|
Folder Structure
Place all files into a folder. You can configure the folder path while setting up the import.Note:MoEngage do not look for files in the root folder or sub-folders.
Setting Up Imports
You can set up the S3/SFTP Imports in the following navigation on the MoEngage Dashboard: Segment -> S3/SFTP Imports. You can also find user-specific imports in the following navigation on the Dashboard: Segment -> Import users.Amazon S3
In order to properly set up S3 Imports, you must give MoEngage relevant permissions and ensure all your files are properly accessible to us. Learn how to set up S3 Imports.SFTP Servers
To learn about the types of authentication methods MoEngage support and the steps to configure, see how to set up SFTP Imports.Imports Dashboard

| Name | Description |
|---|---|
| Name | The name of the import you gave while setting it up. |
| Type | The source of the imports, MoEngage support S3, SFTP, CSV, Snowflake The type of imports, One-Time or Periodic, is also mentioned in the brackets. |
| Custom Segment | If the user import happens in a custom segment, the name of the segment is displayed in this column. |
| Created at | The date and time of the import when it was created. |
| Last Sync Status | The last status of the import. One-Time Imports: The current status of the import.
Recurring Imports: The current status of the latest scheduled import.
|
| Files Processed | The number of files processed to date. |
| Rows Processed | The total number of rows processed to date. |
| Actions | You can choose to View Details, Edit, Duplicate, or Delete an import. |

| Name | Description |
|---|---|
| Scheduled At | The name of the import you gave while setting it up. |
| File Name | The name of the file imported. |
| Status | The status of the import.
|
| Rows in file | The number of data rows present in the file. |
| Events Added | The number of events that were imported from the file. |
| Events Failed | The number of events that failed to import. |
| Users Updated | The number of records identified in your import for existing users that were submitted for potential update in MoEngage. |
| Users Created | The number of records identified in your import for new users that were submitted for potential creation in MoEngage. |
| Users Failed | The number of user records from the import that could not be initially processed due to errors (e.g., invalid format). |
| Aux Data Added | The number of auxiliary data rows added during the import. |
| Aux Data failed | The number of auxiliary data rows failed during the import. |
| Rows Skipped | The number of rows that were skipped during the import. |
| Actions | You can choose “Export File” to download a copy of the file that was imported. You can export files for up to 60 days. |
Triggering Imports
You can trigger imports using one of the following options:- Manually : Periodic imports can be triggered manually from the imports page by clicking Actions Note: Triggering an import within five minutes of the scheduled time may lead to an error.

- API : Imports can be triggered using the File Imports Trigger API. For more information, refer to File Imports Trigger API.
File Limits
The maximum file size that MoEngage supports is 200MB per file. In case of a bigger file, the file must be split into 200MB files each.- Rows: MoEngage allows a maximum of 1M rows in a single CSV/JSON file.
- Columns/Attributes: MoEngage allows 100 columns (attributes) for users and 100 columns (attributes) per event.
- File size: The preferred file size should be less than 200MB (per CSV/JSON) for S3/SFTP and less than 150MB (per CSV) for Manual Uploads.
- File name prefix: The maximum number of characters in the file name prefix should be 100. These limits apply to S3 and SFTP Imports.
- Number of files: In each import run, MoEngage can import a maximum of 250 files.
- Import limits: Refer to the table below.
| Import Type | Users | Events |
| Default Hourly | 600K/hr | 5M/hr |
| Default Daily | Total 14M/day | Total 40M/day |
- The values mentioned above are the default rate limits. If your requirement exceeds these rate limits, you can get in touch with MoEngage to increase them.
- The default hourly rate indicates the total data that can be ingested in an hour and does not indicate the total data limit for the day. The Default Daily limit is the total cap on the data that can be ingested in a day, as indicated in the table above.
- For one-time historical imports, get in touch with the support team to guide you.
- When the rate limits for the imports are breached, you receive a Rate limit breached alert in the Segment-> Imports section on the MoEngage Dashboard.
Best Practices
To ensure your data imports run smoothly and efficiently, consider the following recommendations:- Optimize File Size for Speed: For faster processing, it’s recommended to keep individual import files under 1 million rows.
- Split Large Datasets: If you have a large amount of data to import, splitting it into multiple smaller files can generally result in faster overall processing than uploading a single, huge file.
Frequently Asked Questions
My imports have failed. How do I check what went wrong?
My imports have failed. How do I check what went wrong?
What happens when there is an error in fetching the files from the S3/SFTP folder?
What happens when there is an error in fetching the files from the S3/SFTP folder?
What if a scheduled import adds the data into a recently archived segment?
What if a scheduled import adds the data into a recently archived segment?
How does MoEngage pick up new files in a periodic run?
How does MoEngage pick up new files in a periodic run?
Will a periodic Data import pick up an earlier processed file?
Will a periodic Data import pick up an earlier processed file?
Can a CSV file contain both User and Event data?
Can a CSV file contain both User and Event data?
Can an import be stopped while it's running?
Can an import be stopped while it's running?
How can I stop future runs of a scheduled import?
How can I stop future runs of a scheduled import?
What should I do if an import seems stuck or is taking a long time?
What should I do if an import seems stuck or is taking a long time?
What is NDJSON, and why is it required?
What is NDJSON, and why is it required?
Are compressed files (e.g.,.gz,.zip) supported?
Are compressed files (e.g.,.gz,.zip) supported?
If I change the file type of an existing import, what happens to my mappings?
If I change the file type of an existing import, what happens to my mappings?