Amazon S3

Overview

Amazon Simple Storage Service (Amazon S3) is an object storage service offered by Amazon as part of Amazon Web Services (AWS) that offers the ability to store and retrieve any amount of data, at any time, from anywhere on the web. With this integration you can easily and reliably send high volumes of event data to Split, as well as export your impression data.

Send events to Split

Prerequisites

To connect your AWS S3 bucket to Split, you need:

1. An S3 bucket that will contain files with supported file types and encodings. (Examples: gzip, bzip2, snappy, or LZO compression.)

2. An AWS account with the ability to grant Split permission to read the folder with data in the bucket as well as read/put permission for the folder where the status is expected to be posted.

Create Amazon S3 bucket

(Follow this step only if you do not already have an S3 bucket. If you already have one created, find your Bucket ARN and move to the next step.)

1. Sign in to the AWS Management Console and open the Amazon S3 console.
2. Click Create Bucket.
3. Create a unique Bucket name. All other settings are optional.
4. Click Save. Note your Bucket ARN.

Create SNS topic

1. Navigate to the Amazon SNS console. Click Create Topic.
2. Under Details > Type, select Standard.
3. Create a Name for your SNS topic. (Example: "split-events")
4. Click Save. Note the SNS Topic ARN.
5. Navigate to your SNS topic and click Edit.
6. Navigate to Access policy.
7. Edit the JSON snippet by adding the following snippet as a new statement (do not replace the full snippet.) Be sure to update the JSON snippet with your SNS Topic ARN, your AWS Account ID, and your Bucket ARN.

{
      "Sid": "S3 Events Notification",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "{YourSNSTopicARN}",
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "{YourAwsAccountID}"
        },
        "ArnLike": {
          "aws:SourceArn": "{YourBucketArn}"
        }
     }
}

8. Click Save.

Configure S3 event notifications

1. In the Amazon S3 console, navigate to your S3 bucket.
2. Under Event notifications, click Create event notification.
3. Provide an Event name.
4. Under Suffix, add the suffix for your compression type (example: .parquet.gz or .snappy.parquet.)
5. (Optional) You may also specify a Prefix like "events/".
6. Under Event types, select All object create events.
7. Under Destination > Destination, select SNS topic. Then, under Specify SNS topic, select Choose from your SNS topics. Find your SNS topic created in the previous section. (Optional: you can also enter the SNS topic ARN if you cannot find your topic in the dropdown list.)
8. Click Save changes.

Configure integration in Split

1. In the Split UI, navigate to admin settings, click integrations, select your workspace, and navigate to the marketplace.
2. Click add next to Amazon S3. (You can also click Warehouse under Categories to filter to Amazon S3.)
3. Under Send events to Split, and click Add configuration.
4. Under Select environment, choose which Split environments that will receive S3 data.
5. Under Paste S3 bucket name, provide the name of your unique S3 bucket from above. (Note: we will create it for you if the directory specified does not exist.)
6. Under Paste SNS topic ARN, provide the SNS topic ARN from above.
7. Under Paste status folder location, provide the file path for status files.
8. Click Save.
9. Navigate back to your Amazon S3 integration. Note that Split has generated two fields: Notification endpoint URL and Role ARN.

Update S3 bucket policy for Split access

1. In the Amazon S3 console, navigate to your S3 bucket.
2. Under Edit bucket policy > Bucket policy, replace the JSON with the snippet below. Note that you should update the Role ARN with the value generated by Split, as well as your bucket name as appropriate.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "{role-ARN}"
            },
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::{bucketName}",
                "arn:aws:s3:::{bucketName}/{filePrefix}/*"
            ]
        }, 
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "{role-ARN}"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::{bucketName}/{statusPrefix}/*"
            ]
        }, 
    ]
}

Update SNS topic subscriptions

1. Navigate to the Amazon SNS console, and find your SNS Topic created above. Click Create Subscription.
2. Under Details, provide your Topic ARN.
3. Under Protocol choose HTTPS.
4. Under Endpoint, provide the Notification endpoint URL generated by Split.
5. Click Create subscription. On the confirmation page, confirm that Check status is “confirmed” and Protocol is “https.”

Send Split impressions to S3

Coming soon!