Audience Studio Data Import Specification

Avatar
Ankit Patel
Follow

Please utilize the following Data Ingestion Spec as a template for file imports into Audience Studio.

Currently, there are 3 main types of attributes supported.

While the format of the file is the same between User and Page attributes, please read below for how each one is different from a data collection and usage perspective.

 

Please take a look at the FAQ section below for some common questions.

User Attribute Ingestion

Definition

User Attributes are attributes that do not tend to change over time. These are generally something about the user, rather than what the user has done on the site/app. Examples of user attributes include: 

Attribute Code

Attribute Value

memberType

“free” OR “paid”

gender

“male” OR “female”

 

Format

  • The file needs to be formatted in a CSV format with delimiter as “^” (the CARET character).
  • The file needs to be UTF-8 encoding (or us-ascii)
  • Each line in the data file should contain 2 columns:
    • The first column represents the User ID and must match either:
      • KUID (a krux cookie id) or MAID (idfa, aaid)
      • Client’s first-party User ID that is used in the user matching process (also known as a Bridge Key).  This needs to have been scraped first from the client's web sites and/or apps in order to produce the match table we need to look up when ingesting this import data.
    • The second column contains all the data associated with the user and should be in the following format:
User1234^gender:male;age:18-24;interest:fishing
User2345^gender:female
User3456^age:35-44
User4567^gender:male;interest:fishing,boating
    • Each row can contain an arbitrary number of attribute-value combinations
    • It is not mandatory for values to specified for all attributes for a given user. For example in the above, User2345 does not need to have the attribute ‘age’ sent in the file if one does not exist.
    • Attribute values need to be less than 255 characters
    • If a user is a member of multiple values, they can be comma-separated.
    • If your attribute value contains a comma or a semicolon in the value and you want it to appear as so in the UI, then you must wrap the attribute value in quotes. Example: attribute3:attribute_value1,"attribute value, with a comma"

  • For each line, please provide a FULL list of attributes for that user. User attributes are NOT incremental, so you will need to provide the full view of the user each time you submit the file. Take a look at the example below:

Day

User Attributes Sent in File

Stored in Database

1

Attribute1 = X, Attribute2 = Y

Attribute1 = X, Attribute2 = Y

2

Attribute1 = X

Attribute1 = X


User IDs:

  • If you are planning on sending over multiple types of IDs (KUID, MAID, custom first-party-id (Bridge Key)), you will have to split up the data so that each ID type is sent in its own file. This should also be uploaded into its own folder in the S3 folder. 

 

File Delivery

Compression:

  • Please make an effort to apply LZO or GZIP compression to the files. This will help reduce the file sizes and speed up delivery to S3. Please make sure that you are applying compression to ALL or NONE of the files to avoid any failures.
  • For complete compression instructions, please take a look at our help article: https://konsole.zendesk.com/hc/en-us/articles/214918988-First-Party-Data-Import

File format:

  • Omit any headers in the first row of the file.
  • The name of the file cannot have spaces or special characters. - or _ can be used.
  • File Drop Location:
    • Where name = client’s s3 bucket name (“acme”)
    • Where {date} is in the format yyyy-MM-dd (2018-06-30)
    • Example: s3://krux-partners/client-acme/uploads/userattributes/2018-06-30/
    • s3://krux-partners/client-{name}/uploads/userattributes/{date}
  • File Name:
    • Where yyy = client name (“acme”)
    • Where {date} is in the format yyyyMMdd (20180630)
    • Example: acme_userattributes_20180630.csv.lzo
    • yyy_userattributes_{date}.csv

 

Page Attribute Ingestion

Definition

Page Attributes are attributes that are more contextual or page-level by nature. It’s defined by recency and frequency of user visits on a page. Page attribute examples include:
 

Attribute Code

Attribute Value

category_viewed

“home” OR “garden” OR ...

product_type

“sofa” OR “televisions” OR ...

 

Format

  • The file needs to be formatted in a CSV format with delimiter as “^” (the CARET character).
  • Each line in the data file should contain 2 columns:
    • The first column represents the User ID and must match either:
      • KUID (a krux cookie id) or MAID (idfa, aaid)
      • Client’s first-party User ID that is used in the user matching process (also known as a Bridge Key).  This needs to have been scraped first from the client's web sites and/or apps in order to produce the match table we need to look up when ingesting this import data.
  • The second column contains all the data associated with the user and should be in the following format:
User1234^category_viewed:home;product_type:sofa
User2345^product_type:televisions
    • Each row can contain an arbitrary number of attribute-value combinations
    • It is not mandatory for values to specified for all attributes for a given user
    • Attribute values need to be less than 255 characters
    • If a user is a member of multiple values, they can be comma-separated.
    • If your attribute value contains a comma or a semicolon in the value and you want it to appear as so in the UI, then you must wrap the attribute value in quotes. Example: attribute3:attribute_value1,"attribute value, with a comma"

  • For each line, please provide an INCREMENTAL list of attributes for that user. Page Attributes do not require a “full view” of the user’s past, simply provide the current day’s worth of updates. Take a look at the example below:

Day

Page Attributes Sent in File

Stored in Database

1

Attribute1 = X, Attribute2 = Y

Attribute1 = X, Attribute2 = Y

2

Attribute1 = Z

Attribute1 = {X, Z}, Attribute2 = Y


User IDs:

  • If you are planning on sending over multiple types of IDs (KUID, MAID, custom first-party-id (Bridge Key)), you will have to split up the data so that each ID type is sent in its own file. This should also be uploaded into its own folder in the S3 folder. 

File Delivery

Compression:

  • Please make an effort to apply LZO or GZIP compression to the files. This will help reduce the file sizes and speed up delivery to S3. Please make sure that you are applying compression to ALL or NONE of the files to avoid any failures.
  • For complete compression instructions, please take a look at our help article: https://konsole.zendesk.com/hc/en-us/articles/214918988-First-Party-Data-Import

File format:

  • Omit any headers in the first row of the file.
  • The name of the file cannot have spaces or special characters. - or _ can be used.
  • File Drop Location:
    • Where yyy = client’s s3 bucket name (“acme”)
    • Where {date} is in the format yyyy-MM-dd (2018-06-30)
    • Example: s3://krux-partners/client-acme/uploads/pageattributes/2018-06-30/
    • s3://krux-partners/client-{name}/uploads/pageattributes/{date}
  • File Name:
    • Where yyy = client/brand (“acme”)
    • Where {date} is in the format yyyyMMdd (20180630)
    • Example: acme_pageattributes_20180630.csv.lzo
    • yyy_pageattributes_{date}.csv

Offline Transaction Data Ingestion

Definition

Transaction data is a set of attributes that enable clients to use Transaction Segments. This is information that’s tied to a conversion event; a valid transaction event needs to have the following attributes sent:

Attribute Code

Attribute Value Example

txn_date

“2018-09-18”

txn_price

“80.65” (must be a number, no currency symbols)

txn_qty

“2” (must be a number)

Any_other_attributes (optional)

Any string-based attribute you wish to associate with this transaction, examples include currency, check-in date, isMember, etc

 

Format

  • The file needs to be formatted in a CSV format with delimiter as “^” (the CARET character).
  • Each line in the data file should contain 2 columns:
    • The first column represents the User ID and must match either:
      • KUID (a krux cookie id) or MAID (idfa, aaid)
      • Client’s first-party User ID that is used in the user matching process (also known as a Bridge Key).  This needs to have been scraped first from the client's web sites and/or apps in order to produce the match table we need to look up when ingesting this import data.
  • The second column contains all the data associated with the user and should be in the following format:
User1234^txn_date:2018-09-18;txn_price:150.00;txn_qty:2;ordertype:pickupInStore
User2345^txn_date:2018-09-18;txn_price:25;txn_qty:1;ordertype:online
  • Each row can contain an arbitrary number of attribute-value combinations but  must have at least the required attributes
  • Keys and values are separated with a “:” such as Key:Value
  • Attributes are separated with a “;”
  • If a user is a member of multiple values, they can be comma-separated
  • Attribute values need to be less than 255 characters
  • Transaction Data is incremental, please only send txn_date equal to the date of which you will be delivering the files. For example, if you are sending the file on 2018-06-30, the transactions should also have a txn_date of ‘2018-06-30’

 User IDs:

  • If you are planning on sending over multiple types of IDs (KUID, MAID, custom first-party-id (Bridge Key)), you will have to split up the data so that each ID type is sent in its own file. This should also be uploaded into its own folder in the S3 folder. 

File Delivery

Compression:

  • Please make an effort to apply LZO or GZIP compression to the files. This will help reduce the file sizes and speed up delivery to S3. Please make sure that you are applying compression to ALL or NONE of the files to avoid any failures.
  • For complete compression instructions, please take a look at our help article: https://konsole.zendesk.com/hc/en-us/articles/214918988-First-Party-Data-Import

 File format:

  • Omit any headers in the first row of the file.
  • The name of the file cannot have spaces or special characters. - or _ can be used.
  • File Drop Location:
    • Where {date} is in the format yyyy-MM-dd (2018-06-30)
    • Example: s3://krux-partners/client-acme/uploads/transactiondata/2018-09-18/
    • s3://krux-partners/client-{name}/uploads/transactiondata/{date}
  • File Name:
    • Where yyy = client name
    • Where {date} is in the format yyyyMMdd (20180630)
    • Example: acme_transactiondata_20180630.csv.lzo
    • This step is optional as the filename does not need to be a specific format, but our recommended values are below
    • yyy_transactiondata_{date}.csv

 

Frequently Asked Questions

What is the format for MAIDs?

IDFA: 32D1AC39-356D-4840-AFE3-FA5FAF6A5ABB (8-4-4-4-12) - Typically all CAPS

AAID: 32d1ac39-356d-4840-afe3-fa5faf6a5abb (8-4-4-4-12) - Typically all lowercase

 

Does case matter with the User IDs?

Yes, case matters. USER1234 is not the same as user1234. If supplying a consent file, please make sure that both files have the userid in the same case.

Have more questions? Submit a request

0 Comments

Article is closed for comments.