Third Party Data Provider Integration Guide

1. Introduction

Salesforce Audience Studio supports ingestion and use of data provided by parties other than the direct Audience Studio client. In this case, the data is generally treated as "third-party data". Third-party data is stored in a special account within Audience Studio and can be made available to any Audience Studio client, with agreement of the third-party data provider.    

If you're looking for information about blended segments, please refer to this article:
https://konsole.zendesk.com/hc/en-us/articles/360018650574-Understanding-Blended-Segment-CPMs.

By default, Audience Studio will allow the data to be generally used by all Audience Studio clients.  

Audience Studio supports two types of datasets:

  • data available to multiple clients
  • data available to a single client

You must specify the dataset type when you request creation of the dataset.

If there are any restrictions on what clients can use the data, or if special paperwork is required, please inform Audience Studio.

2. Integration Overview

This section describes the integration model open to third-party data providers.

2.1 Server-to-Server Integration

The Server-to-Server (S2S) integration method involves three steps:

  1. Cookie Matching: Audience Studio and the third-party data provider synch their respective user IDs via "Cookie-Matching".
  2. Data Files: The data provider sends data files to Audience Studio's Amazon S3 account. Daily refreshes are recommended to minimize latency of processing between the data provider system and Audience Studio. Incremental refreshes are recommended to optimize data volumes.
  3. Segment Taxonomy: The third-party data provider sends Audience Studio a meta-file containing: segment ids, names, categories and prices periodically, as it changes.

2.1.1 Cookie Matching

Cookie matching is the process of syncing user IDs between Audience Studio and the data provider. The user match can be initiated by the data provider and/or by Audience Studio. In either case, Audience Studio will keep the user match table.

If you (the Partner) want Audience Studio to initiate this call, use the following process:

  1. Provide Audience Studio with a cookie-match URL
  2. Audience Studio provides you (the Partner) with a user match pixel which you should redirect to from the above pixel. This should be a HTTP 302 redirect to the Audience Studio pixel with your partner user ID.
  3. Audience Studio fires your cookie-match URL on your client's web sites. All cookie-match pixels are frequency-capped to 3 per-day per-user.

If you (the Partner) want to initiate this call, use the following process:

  1. Contact the Salesforce Audience Studio Solutions team to get the Audience Studio pixel location that you should redirect to after including your user ID.
  2. Audience Studio provides you (the Partner) with a user match pixel that you should fire from all across your network.
  3. The above pixel fire should include your user ID.

2.1.2 Persistent Identifiers

Audience Studio supports delivery of data tied to persistent identifiers.

Mobile data can be delivered to Audience Studio using IDFAs and AAIDs. (Link "IDFA" to https://developer.apple.com/documentation/adsupport/asidentifiermanager and "AAID" to https://support.google.com/googleplay/android-developer/answer/6048248?hl=en)

Client-specific data can also be delivered using a client's first party ID.

2.1.3 Data Files

The data files present the format in which partners should provide data to Audience Studio keyed off the partner's user ID.

Audience Studio will provide an Amazon S3 location and the format to ingest these data files from the partner. Please see the following sections for specific information about the structure and details of the data file.

The Audience Studio standard integration supports incremental refresh of the users in the daily files. Please be advised that the associated segments are always updated in full and not incrementally. This combination of incremental user refresh and full segment refresh enables the flexibility to add and delete segments for a user.

Example:

Contents of Day1 file:

user_A^S1,S2,S3
user_B^S3,S4,S5

Contents of Day2 file:

user_A^S4,S5
user_C,^S1,S3,S5

Data will reflect the following after final file delivery:

user_A^S4,S5
user_B^S3,S4,S5
user_C^S1,S3,S5

To comply with global privacy regulations, users are deleted by the application every 90 days. 

If you want to retain the user data, you have to resend each user at least once every 90 days. Data partners can either send a full refresh of all user data every 90 days, or resend specific users that haven't been sent in the last 90 days.

2.1.4 Taxonomy Files

Taxonomy files are used to send information about segments sent by the data partner. This can include names, hierarchy, and prices.

Taxonomy files are updated as a full refresh. The latest taxonomy file overwrites all previous segment information. Because of the full refresh,  taxonomy files must include information on all active segments. Please see the following sections for specific information about the structure and details of the taxonomy file.

 

3. Setup

Your setup will be managed by a dedicated Salesforce team and during your initial setup:

  • You will receive an automated email to set up an S3 folder and download access credentials;
  • You will also work with the Salesforce team to provide details for the data delivery destination (both for initial setup and for subsequently added data delivery destinations)

In the rest of this document, we will use krux-partners/partner-xxx as a placeholder for the dedicated folder name. Please make sure to replace this placeholder with your unique S3 folder path.

4. Directory Structure

Audience Studio supports multiple data delivery destinations with each partner. Each data delivery destination can be set up to be available to all Audience Studio clients or restricted to a single Audience Studio client. Note that we do NOT support data delivery destinations available to a subset of clients. And as a reminder, set up of a new data delivery destination requires coordination with the Salesforce Audience Studio team.

For every data delivery destination $destination, your S3 folder must include a single taxonomy file in the following directory:

s3://krux-partners/partner-xxx/taxonomies/$destination/$day/


Example:

s3://krux-partners/partner-foo/taxonomies/data_set_bar/2019-08-10/


And one or more data files at:

s3://krux-partners/partner-xxx/$destination/$datasource/$day/$devicetype/$filetype/

 

Where:

 Directory Elements  Description
$destination must be exactly the same in the taxonomy and data file folders
$datasource another level of organization, often used by data partners as a way to organize data from different internal sources
$day format is yyyy-mm-dd and needs to be dated the day it is uploaded. Data processing jobs look at data delivered over multiple days and treat data delivered on a later date as the most recent information.
$devicetype this is the way to deliver data associated with existing segment IDs. These folders can be any name and we recommend you use commonly used names such as AAID, IDFA, MAID, FPID, CookieID, KUID. Each segment ID must be alpha-numeric, with no special characters, and must be less than 255 characters long.
$filetype

format is one of full or incr. Full is used for a complete overwrite of the data, and incr is used for an incremental update of data. Every $destination can use a combination of these types as appropriate to guide updates of audience data in the Audience Studio.


For Client A:

s3://krux-partners/partner-yourcompany/client_a/


For Client B:

s3://krux-partners/partner-yourcompany/client_b/

Example:

s3://krux-partners/partner-foo/data_set_bar/app_data/2019-08-10/IDFA/incr/
s3://krux-partners/partner-foo/data_set_bar/app_data/2019-08-10/AAID/incr/
s3://krux-partners/partner-foo/data_set_bar/crm_data/2019-08-10/Cookie/full/

Review the following considerations before you create your folder structure:

  • All directories in the path specified above are required.
  • All user data delivered into a $devicetype folder must be associated with IDs of the same device type. At the time of setup, you will specify a list of folders you expect to use for that data delivery destination and the ID type that will be delivered to each of these. This is used by Audience Studio to link to the appropriate user match tables.
  • When using a combination of full and incr data delivery, Audience Studio will look back to the last full file delivered and process all incremental files delivered after that day.
  • When using incr data delivery alone, Audience Studio will process all incremental data delivered over the last 90 days.
  • incr file delivery allows you to send user data only on those users with new/changed attributes rather than send data on all users all the time.

Note: Data on a given user does not need to be sent unless any of the segments/attributes associated with the user have changed. But when you submit ANY attributes on a user you must submit ALL attributes associated with that user (changed and unchanged) as data is overwritten at the user level, not the attribute level.

The format for data files is:

user_id^S1, S2, S3
user_id = The data provider user ID
table S#_ = IDs of data provider segments that match names in taxonomy file (required)

 

Note: The segment list must be included with every delivery - even incremental user data.
$datasource can be used to deliver independent groups of segments for the same user. If incr file delivery is used with multiple data sources, the requirement to resend all segments for a user is applicable only within the datasource. In other words, if a user's segments change within one datasource, the user needs to be resent only in that datasource Other data sources do not need to resend information for that user.

5. Taxonomy File Guidelines

The taxonomy file is the data dictionary. It is the file that defines the metadata for the segment IDs in your data delivery. As mentioned above, the taxonomy (.csv) files must be added to the following directory:

s3://krux-partners/partner-xxx/taxonomies/$destination/$day/

The data in the taxonomy file you upload must be in the following format: 

{category-1, ... category-n}, segment_name, segment_id, price

Review the following considerations while creating your taxonomy file:

  • Each taxonomy file you upload will completely overwrite the previous taxonomy. Hence if you are adding or deleting segments in your $destination, you need to also include all the segments that are not being modified.
  • Each $destination requires a single taxonomy file. If you have multiple $datasource with distinct segments in your data delivery destination, you will need to combine the segment definitions across these into a single taxonomy file.
  • Each line in your upload file should have a minimum of 4 columns, and at least one category. You can have as many categories as you need, as long as the last 3 columns are segment name, segment id, and price. 
  • Each segment entry in your upload file needs to be on its own line.
  • Do not include trailing commas at the end of a line.
  • Your taxonomy file needs to be in comma separated (.csv) format and should not be compressed.
  • Do not include headers in your upload file.
  • If you have text fields that contain a comma, the fields need to have quotes around them. Example: "$1,000-$2,000"
  • The price is the CPM and this field in your upload file is stored as a numeric value. Do not put quotes around it.
  • The characters in your upload file must be in UTF-8 format only.

Example Taxonomy Rows:

Foo Automotive Data,Car Rental,Personal Use,Company Test1,23456,0.75
Foo Automotive Data,Car Rental,Personal Use,Any Company in Category,34567,0.75
Foo Automotive Data,Car Rental,Business Use,Company Test1,45678,0.75
Foo Automotive Data,Car Rental,Business Use,Company Test2,67890,0.75
Foo Luxury Cars,12345,1.00

6. Data File Guidelines

The data files specify the segment IDs associated with individual users. Post processing in Audience Studio, these determine the user populations for delivered segments. As mentioned above, the data file folder structure must confirm to:

krux-partners/partner-xxx/$clientname/$datasource/$day/$devicetype/$filetype/

The contents of the data file you upload must be in the following format: 

userid^{segmentid-1,... segmentid-n}

Review the following considerations while creating your data file:

  • The segment IDs in your data files need to match the segment IDs that were delivered in your associated taxonomy file. Segment IDs that are not found in the associated taxonomy file will not be imported. Each segment ID must be alpha-numeric, with no special characters, and must be less than 255 characters long.
  • All segments associated with a user must be included in the data file. If a segment needs to be added or removed for a user, a full list of the segments associated with the user must be resent.
  • The userid in your data file can be the partner cookie ID, a mobile device ID, first party ID or other supported Audience Studio ID. However, all line items must contain the same type of ID you specified for the upload folder. In other words, you cannot deliver segments for cookie IDs and mobile device IDs in the same $devicetype folder.
  • You can upload multiple compressed part files. If you upload compressed files, the size needs to be under 1GB.
  • You can upload multiple uncompressed part files. If you upload uncompressed files, the files can be any size. 

7. Updates to Taxonomy and Data

Data Partners under standard integration can update their taxonomy and data files at any time. Audience Studio automatically uses the last delivered taxonomy file and combines data files as appropriate.

Data Partners that are not yet migrated to the standard integration will need to request updates to taxonomy files through Support. Updates are limited to once a quarter.  

To request an update, submit your .csv file via the Submit a Request form, or by sending to the Salesforce Audience Studio Support team at kruxhelp@krux.com.

8. Process to Send Files to Private/Custom Directories 

To upload files to your custom directory, email a Zendesk Ticket to CST (kruxhelp@krux.com) that includes the following information:

  1. the S3 location of your data files
  2. the S3 location of your taxonomy or taxonomy included in the ticket
  3. type of data: 1st party, 2nd party or 3rd party
  4. billable or non-billable
  5. name of the data provider (client sees in the UI)
  6. customer to be enabled

9. Usage Report

Audience Studio customers and activation partners are required to provide monthly usage reports based on targeted impressions.

An activation partner may provide usage reporting on behalf of mutual customers. If an activation partner does not choose to provide usage, then each customer is required to provide usage for each activation partner.

We usually provide reports to Data Providers by the 30th day following the previous month.

If a customer does not provide usage reports by the deadline each month, they are restricted from activating billable data.

Customers may provide historical / late usage to lift activation restrictions and we support up to three months of usage (current month and two prior months).

Additional details can be found in the following help center articles.

https://konsole.zendesk.com/hc/en-us/articles/360018799514

https://konsole.zendesk.com/hc/en-us/articles/360021492374-Self-Reported-Usage-File-Upload-and-Restricted-Activation

https://konsole.zendesk.com/hc/en-us/articles/360022469353

10. Deleting Data

To delete a segment, you must deliver a new taxonomy file that no longer includes that segment.

To delete segments associated with a user, you must deliver that user in a data file with the latest group of segments associated with that user.

To delete all of a user’s data, you must either deliver that user in an incremental data file with all segments removed or deliver a full data file without that user included. 

Need Help?

If you need help with Third-party Data Provider Integration, don't hesitate to contact the Salesforce Audience Studio Support team at kruxhelp@krux.com.

Have more questions? Submit a request

0 Comments

Article is closed for comments.