Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service offered by Amazon as part of Amazon Web Services (AWS) that offers the ability to store and retrieve any amount of data, at any time, from anywhere on the web. With this integration you can easily and reliably send high volumes of event data to Split, as well as export your impression data.

Send events to Split

The following describes how to send events to Split.

Prepare your S3 bucket and event files

To connect your AWS S3 bucket to Split, you need:

• An S3 bucket that contains files with supported file types and encodings, e.g. gzip, bzip2, snappy, or LZO compression.
• An AWS account with the ability to grant Split permission to read the folder with data in the bucket as well as read/put permission for the folder where the status is expected to be posted.

File restrictions:

• Compression examples: gzip, bzip2, snappy, or LZO compression.

• File format: parquet only

• Schema:

  - environmentId: [String] Identifier for the environment in Split
  - trafficTypeId: [String] Identifier for the traffic type in Split
  - eventTypeId: [String] Name for the type of event
  - key: [String] Identifier for the "user" or "system" triggering or associated with the event
  - timestamp: [Long] The time when the event took place, in milliseconds past epoch
  - value: [Double, Nullable] A numeric value associated with this event
  - properties: [Map<String,String>, Nullable] Arbitrary properties data providing context to the event

To send the same event into multiple environments within Split, duplicate the events with different environment IDs.

Create Amazon S3 bucket

Note: Follow this step only if you do not already have an S3 bucket. If you already have one created, find your Bucket ARN and move to the next step.

1. Sign in to the AWS Management Console and open the Amazon S3 console.

2. Click Create Bucket.

3. Create a unique bucket name. All other settings are optional.

4. Click Save. Note your Bucket ARN.

Create an SNS topic

1. Navigate to the Amazon SNS console and click Create Topic.

2. Under Details > Type, select Standard.

3. Create a Name for your SNS topic, e.g. "split-events".

4. Click Save. Note the SNS Topic ARN.

5. Navigate to your SNS topic and click Edit.

6. Navigate to Access policy.
7. Edit the JSON snippet by adding the following snippet as a new statement. Do not replace the full snippet. Be sure to update the JSON snippet with your SNS Topic ARN, your AWS Account ID, and your Bucket ARN.

{
      "Sid": "S3 Events Notification",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "{YourSNSTopicARN}",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "{YourAwsAccountID}"
        },
        "ArnLike": {
          "aws:SourceArn": "{YourBucketArn}"
        }
     }
}

8. Click Save to save your SNS topic.

Configure S3 event notifications

1. In the Amazon S3 console, navigate to your S3 bucket.

2. Under Event notifications, click Create event notification.

3. Enter an Event name.

4. Under Suffix, add the suffix for your compression type, e.g., .parquet.gz or .snappy.parquet.

5. Optionally specify a Prefix like "events/".

6. Under Event types, select All object create events.

    

   
   

7. Under Destination > Destination, select SNS topic. Under Specify SNS topic, select Choose from your SNS topics. Find your SNS topic created in the previous section.

8. Optionally enter the SNS topic ARN if you cannot find your topic in the list.

9. Click Save changes.

Configure integration in Split (inbound)

1. In the left hand navigation, click the initials at the top left, click Admin settings and then Integrations.

2. Select your workspace, and navigate to the Marketplace.

3. Click Add next to Amazon S3. You can also click Warehouse under Categories to filter to Amazon S3.

4. Under Send events to Split, click Add configuration.

5. Under Select environment, select which Split environments that will receive S3 data.

6. Under Paste S3 bucket name, provide the name of your unique S3 bucket from above.
   Note: We will create it for you if the directory specified does not exist.

7. Under Paste SNS topic ARN, provide the SNS topic ARN from above.

8. Under Paste status folder location, provide the file path for status files.

8. Click Save.
9. Navigate back to your Amazon S3 integration. Split has generated two fields: Notification endpoint URL and Role ARN.

Update S3 bucket policy for Split access (inbound)

1. In the Amazon S3 console, navigate to your S3 bucket.
2. Under Edit bucket policy > Bucket policy, replace the JSON with the snippet below. You should update the Role ARN with the value generated by Split, as well as your bucket name as appropriate.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "{role-ARN}"
                },
                "Action": [
                    "s3:Get*",
                    "s3:List*"
                ],
                "Resource": [
                    "arn:aws:s3:::{bucketName}",
                    "arn:aws:s3:::{bucketName}/{filePrefix}/*"
                ]
            }, 
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": "{role-ARN}"
                },
                "Action": [
                    "s3:PutObject",
                    "s3:PutObjectAcl"
                ],
                "Resource": [
                    "arn:aws:s3:::{bucketName}/{statusPrefix}/*"
                ]
            }, 
        ]
    }
    ]

Update SNS topic subscriptions

1. Navigate to the Amazon SNS console, and find your SNS Topic created above. Click Create Subscription.

2. Under Details, enter your Topic ARN.

3. Under Protocol, select HTTPS.

4. Under Endpoint, enter the Notification endpoint URL generated by Split.

5. Click Create subscription. On the confirmation page, confirm that Check status is "confirmed" and Protocol is "https."

Send Split impressions to S3

Prepare your S3 bucket

To connect your AWS S3 bucket to Split, you need:

• An S3 bucket destination already created.
• An AWS account with the ability to grant Split permission to read,put, and remove data in the bucket. Note: We will only remove transactional files created during the transfer.

Configure integration in Split (outbound)

1. In the left hand navigation, select the initials at the top left, select Admin settings and click Integrations.
2. Select your workspace, and navigate to the marketplace.
3. Click Add next to Amazon S3. You can also click Warehouse under Categories to filter to Amazon S3.
4. Under Send Impressions to S3, click Add configuration.
5. Under Select environment, select the Split environments that will receive S3 data.

6. Under Paste S3 bucket name, enter the name of your unique S3 bucket name.

7. Under Provide the write location in your S3 bucket, optionally put the S3 prefix you'd like the impressions files to be saved to. If no value is provided, files are saved to the root directory of your provided S3 bucket.

8. Under Select file format, select the desired format for your impressions files.

9. Under Select file compression type, select the desired file compression format for your impression files.

   Note: The list only displays compression methods that are compatible with your chosen file format.

10. Click Save. Note your role ARN generated by Split.

Update S3 bucket policy for Split access (outbound)

1. In the Amazon S3 console, navigate to your S3 bucket.
2. Under Permissions, select Edit bucket policy then Bucket policy
3. Replace the JSON with the snippet below. Be sure to replace the following parameters in the snippet:
   • {role-ARN-to-be-provided-to-you-by-Split} with the Role ARN generated by Split above.
   • {bucketName} with your S3 bucket name.
   • {writeLocationPrefix} with the write location prefix as appropriate. If you left the write location section blank in the Split configuration step, and intend to write to the root directory, you must update the following line:
   • "Resource": "arn:aws:s3:::{bucketName}/{writeLocationPrefix}/" > "Resource": "arn:aws:s3:::{bucketName}/".
4. Optionally, if you later update the integration to include a write location prefix, or change the write location in any way, we recommend changing the policy to the more restricted line provided below to include the {writeLocationPrefix__}/ *

 {
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Principal": {
               "AWS": "{role-ARN-to-be-provided-to-you-by-Split}"
           },
           "Action": "s3:ListBucket",
           "Resource": "arn:aws:s3:::{bucketName}"
        },
        {
           "Effect": "Allow",
           "Principal": {
               "AWS": "{role-ARN-to-be-provided-to-you-by-Split}"
           },
           "Action": [
               "s3:PutObject",
               "s3:PutObjectAcl",
              "s3:DeleteObject"
           ],
           "Resource": "arn:aws:s3:::{bucketName}/{writeLocationPrefix}/*"
       }
   ]
}

SSE_KMS Support (Optional)

If your S3 bucket is encrypted using an AWS KMS key (SSE-KMS), you need to update the policy to allow Split's AWS ARN to encrypt data using the KMS key you have chosen.) If your S3 bucket is using an Amazon S3 key for encryption (SSE-S3), then the following is not necessary.

1. In the Amazon S3 console​, navigate to the Key Management Service (KMS) dashboard.
2. Navigate to the KMS key being used for your encrypted S3 bucket.
3. Navigate to the KMS key policy and make sure the following actions are included in your existing key policy:

{
   "Version": "2012-10-17",
   "Id": "cross-account-policy",
   "Statement": [
       {
           "Sid": "KMSOwner",
           "Effect": "Allow",
           "Principal": {
               "AWS": "arn:aws:iam::{your-aws-account-id}:root"
           },
           "Action": "kms:*",
           "Resource": "*"
       },
       {
           "Sid": "KMSCrossAccount",
           "Effect": "Allow",
           "Principal": {
               "AWS": "{role-ARN-to-be-provided-to-you-by-Split}"
           },
           "Action": [
               "kms:DescribeKey",
              "kms:GenerateDataKey",
   "kms:Decrypt"
           ],
           "Resource": "*"
      }
    ]
}