Use the Agent for AWS Lambda to Secure Data in AWS S3 Bucket
  • 17 Oct 2020
  • 5 Minutes To Read
  • Print
  • Share
  • Dark
    Light

Use the Agent for AWS Lambda to Secure Data in AWS S3 Bucket

  • Print
  • Share
  • Dark
    Light

SecureCircle transparently protects data at rest, in use, and in-transit. In order to protect data initially, SecureCircle Administrators can establish ingestion points where data is automatically protected as it ingresses. Using the SecureCircle Agent for AWS Lambda, data that ingresses into an Amazon S3 Bucket can be automatically protected using SecureCircle.

Register an Endpoint to Obtain an Endpoint ID for Use with AWS Lambda
In order to authorize an AWS Lambda function to access or protect data in a SecureCircle Circle, an endpoint ID issued by SecureCircle is required. SecureCircle administrators take the following steps to create a new endpoint ID.

  1. In the Circles menu in the SecureCircle Admin UI, select one or more Circles, then select "Actions" (or right-click) and select "Invite User".
  2. In the Invite User prompt, enter a non-routeable email address. For example: sc-lambda-agent@securecircle.local

lambda1.png

  1. Manage one of the Circles to which the user was invited by clicking on the Circle's name. In the Circle management screen, select the new user's row and find the invitation ID in the Description section
    lambda2.png

  2. Issue the following API call (this example uses curl)

INVITATION_ID=your_invitation_id_goes_here SERVER_HOST=example.securecircle.local curl -v -XPOST -H 'Content-Type: application/json' https://$SERVER_HOST/tracker/v3/endpoints/register --data '{
    "identity":{"ads":{"user":"","groups":[]}},
    "invitationId":"'$INVITATION_ID'",
    "hardwareId":"8C0C60AA-2564-42AD-BC6E-E1B6DCA30B81",
    "endpointName":"sc-lambda-agent",
    "endpointType":4,
    "versionInfo":{"version":"1.0.0","module":"sc-lambda-agent","rev":"1.0.0","comments":""}
}'
Note

INVITATION_ID (from step #3) and SERVER_HOST must be entered based on your server's configuration. The hardwareId field is also configurable: set it to any UUID that is preferred, or keep the UUID from this example.

  1. Navigate to the Devices page and select the "sc-lambda-agent" device. Right-click on the row and select "Confirm Device" to allow it access to SecureCircle. Then, note the "Device ID" in the Description. This is the endpoint ID that will be used with AWS Lambda.

lambda3.png

Determine the Circle ID with which to Protect Data

Data will be protected in the scope of a Circle. The Circle's ID is used by the AWS Lambda function to identify the Circle in which to protect data. A Circle's ID may be found by navigating to the Circles menu in the SecureCircle Admin UI, selecting the row corresponding to a Circle, and then finding the Circle ID field in the Description section.
lambda4.png

Create a New AWS Lambda Function

SecureCircle recommends referring to the latest AWS Lambda and S3 documentation in order to stay current with the best practices for these services. The following AWS documentation specifically covers Lambda and S3: https://docs.aws.amazon.com/lambda/latest/dg/with-s3.html. The following are common items created/settings when creating an AWS Lambda function that interacts with AWS S3.

Create an IAM Role for the Lambda Function to Assume

The latest AWS documentation describes Lambda execution roles: https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html. The following steps are an example of how to create an Lambda Execution Role for a Lambda that has permissions to interact with Objects in AWS S3.

  1. In the IAM Service, create a new role for use with AWS Lambda.
  2. Add an Inline Policy to the role, similar to the following:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::demo-bucket/sc-lambda-agent-demo/*",
                "arn:aws:s3:::demo-bucket"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:HeadBucket",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

Note: in the above example, the Lambda IAM Execution Role has permissions to access Objects in the "demo-bucket" (replace this with the bucket name that the Lambda function will be interacting with, and add more bucket ARNs to the Resource list as needed). It may also be advisable to narrow the Resource scope of the "logs:*" permissions to allow the Lambda only access to its own, dedicated Log Group.

Create a Lambda Function Based on the s3-get-object-python Blueprint

The s3-get-object-python Lambda Blueprint selects the correct Python runtime (Python 3.x) and walks the user through configuring the appropriate triggers for S3. The default code can be used during creation as it will be overridden with the SecureCircle-provided Lambda Agent Zip Package.

  1. In the AWS Lambda Console, select "Create Function".
  2. Select "Use a Blueprint" and filter and select the "s3-get-object-python" item.
  3. Use the following table as a reference for configuring the settings in the final page of the "Create Function" dialog. When finished, select "Create Function"
Item Description Example
Basic Information → Function name The name of the Lambda function. sc-s3-encrypting-function
Basic Information → Execution role The IAM role the Lambda function will assume during execution. Use the existing IAM role created in the "Create an IAM Role for the Lambda Function to Assume" section of this document. Use an existing role: sc-s3-encrypting-function-iam-role
S3 trigger → Bucket The bucket whose events will trigger the Lambda function to run. For example, the S3 bucket to which unprotected files may be PUT demo-bucket
S3 trigger → Event type The S3 events that should trigger the Lambda. For example, "All object create events" will cause the Lambda to run whenever a new object is created. All object create events
S3 trigger → Prefix The prefix (similar to folder/path) that objects must have to trigger the Lambda. files/unprotected-file-upload-area/
S3 trigger → Suffix The suffix (similar to file extension) that objects must have to trigger the Lambda 'leave blank'
S3 trigger → Enable trigger Whether or not the trigger should start monitoring events and executing the Lambda immediately 'checked'
Lambda function code The blueprint's Lambda function code in Python. Leave this as-is (it will be replaced) 'leave as-is'

Configure the Lambda Function for the SecureCircle Lambda Agent

Once the blueprint-based Lambda Function is created, the code must be overwritten and environment-specific settings applied. The following steps provide an example:

  1. Navigate to the newly-created Lambda Function's "Configuration" page.
  2. In the "Function code" section, select "Upload a .zip file" for the "Code entry type" and click the Upload button to browse and select the SecureCircle Lambda Agent Zip package.
  3. Use the following table as a reference for configuring the settings for the remainder of the Lambda "Configuration" page.
header header header
Environment variables → CIRCLE_ID The Circle ID (see the "Determine the Circle ID with which to Protect Data" section of this article) corresponding to the Circle in which data in the S3 bucket will be protected e1af932b-efbc-4d90-af75-1ad4659db5c7
Environment variables → ENDPOINT_ID The Endpoint ID (see the "Register an Endpoint to Obtain an Endpoint ID for Use with AWS Lambda" section of this article) which is authorized to protect data in the Circle. 960845ff-f6a2-4796-9905-ff5cd5f12ff2
Environment variables → REPLACE_KEY_FROM The string to match and replace in a newly-created S3 Object's prefix. This is optional if DESTINATION_BUCKET is set. files/unprotected-file-upload-area/
Environment variables → REPLACE_KEY_WITH The string with which REPLACE_KEY_FROM will be replaced. This is required if REPLACE_KEY_FROM is set. files/protected-files/
Environment variables → DESTINATION_BUCKET The new bucket to which protected data will be put. This is optional if REPLACE_KEY_FROM is set. 'not configured, do not create key or value'
Environment variables → SERVER_HOST The hostname/FQDN of the SecureCircle Server example.securecircle.local
Basic settings → Timeout The amount of time for which a single Lambda Function execution is allowed to run. This should be enough time to completely download, process, and upload an S3 Object. 15 minutes
Was This Article Helpful?