Transforming Your Data with Amazon S3 Object Lambda

Services Covered

  • S3 S3
  • lambda Lambda

Lab description

Transforming Your Data with Amazon S3 Object Lambda.

Learning Objectives

Create amazing Lambda
Create an Amazon S3 Access Point
Create Object Lambda Access Point

Lab date

25-12-2021

Prerequisites

AWS account

Lab steps

  1. Create a S3 bucket.
  2. Click the Access Points tab. Configure the following properties for your access point: Access point name: Enter lab-ap Network origin: Select Internet Block Public Access settings for this Access Point: Unselect the checkbox for Block all public access.
  3. Click Object Lambda Access Points in the left-hand navigation menu: Object Lambda Access Point name: Enter lab-olap Supporting Access Point: Click Browse S3 In the resulting Choose supporting Access Point popup, select lab-ap, then click Choose supporting Access Point.

    • Invoke Lambda function: Ensure Choose from functions in your account is selected
    • Lambda function: Select the CsvToJsonConverter function
  4. Navigate to Lambda and create CsvToJsonConverter in Python: 
    import boto3
import urllib3
import csv
import json

def lambda_handler(event, context):

   object_get_context = event["getObjectContext"]
   request_route = object_get_context["outputRoute"]
   request_token = object_get_context["outputToken"]
   s3_url = object_get_context["inputS3Url"]

   # Get object from S3
   http = urllib3.PoolManager()
   response = http.request('GET', s3_url)
   original_object = response.data.decode('utf-8').splitlines()
   csv_reader = csv.DictReader(original_object)
   data = {}

   for rows in csv_reader:
       id = rows['Identifier']
       data[id] = rows

   json_object = json.dumps(data, indent=4)

   # Write object back to S3 Object Lambda
   s3 = boto3.client('s3')
   s3.write_get_object_response(
       Body=json_object,
       RequestRoute=request_route,
       RequestToken=request_token)

   return {'status_code': 200}

Lab files

Acknowledgements

By : LU Tags: