Tommi's Scribbles
Using AWS Lambda Layers with node.js
- Published on 2022-02-12
AWS Lambdas are a great feature to develop modern "server" type applications. The server is in quotes as AWS Lambdas are in essence serverless technology. Lambdas on AWS are the smallest microservice building block. With layers, you can have a lot of power making lambdas a nice alternative to containers. This write-up shows an example on what you can do with layers.
NOTE: For the reminder of the article, lambda will refer to the AWS Lambda, not a language lambda.
Getting started
The purpose of the lambda layers is to allow you to reuse common code, such as libraries. Another use case, and one we are doing today is adding common API functionality for all the lambdas to use. AWS has good instructions on how to add the layers to your functions, so we won't touch that topic here.
Key thing with the layer is that it should follow a certain directory structure. So let's create a folder called layer. In there, we add another folder called nodejs. So now we have layer/nodejs structure, where we add the following files:
- api.js
- package.json The contents of the package.json are as follows:
{
"name": "api-layer",
"version": "1.0.0",
"private": true,
"dependencies": {
"aws-sdk": "latest",
"bcryptjs": "latest",
"jsonwebtoken": "latest"
}
}
After you create all this, you can run npm install to install the packages and automatically create the rest that is needed for the layer. With the initial setup out of the way, we can start editing the api.js.
Example Layer API class
We will start by including the AWS sdk. This way, instead of having all your lambda functions having to include the AWS SDK, you only include it the layer, thus reducing the size of your actual lambda.
const AWS=require ( "aws-sdk" )
We will also add some libraries for authentication API help. We will also define a key here, which you ideally should not store in your code as is. If you check my Lambda Authenticator for API gateway article, you will see the same key used there. This should hint you of its use.
const bcrypt=require ( 'bcryptjs' ) const jwt=require ( "jsonwebtoken" ) const secretKey='aSUPERsecretKey'
We can then start constructing the actual layer. We will define a class that the lambdas can then create if they need access to any AWS features or helpers we define here. For example purposes, we will call the class MyLambdaAPI. Feel free to use a more specific name to your purposes.
class AppAPI
{
constructor ()
{
this.dbclient=new AWS.DynamoDB.DocumentClient ()
this.roleMap=new Map ( [["admin", 1], ["superuser", 10], ["user", 100]] )
}
}
We start with a simple constructor. Note that the variables could be declared private to the class as you don't really need them outside the class.
We also need to export the module for the lambdas to use, so add this to the end of the file.
module.exports=PortalAPI
Now we can start adding functions. As we included bcrypt, and many systems have login functionality, let's add an authentication function inside the class (that is, before the last }). We can also add functionality to generate the hashes used by authentication.
authenticate=function( plain, hashed )
{
return bcrypt.compareSync ( plain, hashed )
}
createHash=function( hashable )
{
let salt=bcrypt.genSaltSync ( 10 )
return bcrypt.hashSync ( hashable, salt )
}
Another common requirement is working with AWS resources, such as S3 and DynamoDB. Let's say you use DynamoDB in a similar way to a traditional relational database. There are good write-ups on the topic, and as the design itself falls outside the scope of this article, I will not cover creating a relational database on DynamoDB here. The key point for such a setup is using two columns, primary key and secondary key, to store the relations.
With the API, we can create an easy-to-use interface to DynamoDB for our Lambdas to utilize in such a scenario. We do that by adding the following functions to the class defined before. Note we call the primary key pk, and secondary key sk.
dynamoDBAdd=async function( tableName, item )
{
let result=false
await this.dbclient.put ( {
"TableName": tableName,
"Item": item
} )
.promise ()
.then ( ( success ) =>
{
result=true
console.log ( "DynamoDB create success: " + JSON.stringify ( success ) )
} )
.catch ( error =>
{
console.error ( "DynamoDB create error: " + error )
} )
return result
}
dynamoDBDelete=async function( tableName, pk )
{
let result=false
let params={
TableName: tableName,
KeyConditionExpression: "#pk = :pkvalue",
ExpressionAttributeNames: {
"#pk": "pk"
},
ExpressionAttributeValues: {
":pkvalue": pk
}
}
await this.dbclient.query ( params )
.promise ()
.then ( async success =>
{
console.log ( "DynamoDB delete: Found entries for " + pk + ": " + JSON.stringify ( success ) )
// TODO: Add batching if counts start going upwards 25
if ( success.Count > 0 )
{
for ( let i=0; i < success.Count; i++ )
{
let itemParams={
"TableName": tableName,
"Key": {
"pk": `${ success.Items[ i ].pk }`,
"sk": `${ success.Items[ i ].sk }`
}
}
console.log ( "DynamoDB Delete: Iterating item: " + success.Items[ i ] )
await this.dbclient.delete ( itemParams )
.promise ()
.then ( success =>
{
console.log ( "DynamoDB Delete: Success for " + itemParams + ": " + success )
result=true
} )
.catch ( error => console.error ( "DynamoDB Delete error: " + error ) )
}
}
else
{
console.warn ( "Deleting item " + pk + ", but no items found." )
result=true
}
} )
.catch ( error => console.error ( "DynamoDB error: " + error ) )
return result
}
dynamoDBDeleteRelation=async function( tableName, pk, sk )
{
let result=false
let params={
"TableName": tableName,
"Key": {
"pk": `${ pk }`,
"sk": `${ sk }`
}
}
await this.dbclient.delete ( params )
.promise ()
.then ( success =>
{
console.log ( "DynamoDB Delete relations success for " + params + ": " + success )
result=true
} )
.catch ( error => console.error ( "DynamoDB Delete relations error: " + error ) )
return result
}
dynamoDBGetItem=async function( tableName, pk )
{
let result=null
let params={
TableName: tableName,
KeyConditionExpression: "#pk = :pkvalue",
ExpressionAttributeNames: {
"#pk": "pk"
},
ExpressionAttributeValues: {
":pkvalue": pk
}
}
await this.dbclient.query ( params )
.promise ()
.then ( async success =>
{
console.log ( "DynamoDB query: " + pk + " entry found: " + JSON.stringify ( success ) )
result=success
} )
.catch ( error => console.error ( "DynamoDB Query Error: " + error ) )
return result
}
dynamoDBGetItems=async function( tableName, sk )
{
let params={
TableName: tableName,
IndexName: "SortKeyGSIIndex",
KeyConditionExpression: "#sk = :skvalue",
ExpressionAttributeNames: {
"#sk": "sk"
},
ExpressionAttributeValues: {
":skvalue": sk
}
}
let result=null
await this.dbclient.query ( params )
.promise ()
.then ( success =>
{
console.log ( "DynamoDB query items found: " + JSON.stringify ( success ) )
result=success
} )
.catch ( error => console.error ( "DynamoDB query error: " + error ) )
return result
}
dynamoDBGetRelations=async function( tableName, pk, sk )
{
let params={
TableName: tableName, // IndexName: "SortKeyGSIIndex",
KeyConditionExpression: "#pk = :pkvalue and begins_with(#sk, :skvalue)",
ExpressionAttributeNames: {
"#pk": "pk",
"#sk": "sk"
},
ExpressionAttributeValues: {
":pkvalue": pk,
":skvalue": sk
}
}
let result=null
await this.dbclient.query ( params )
.promise ()
.then ( success =>
{
console.log ( "DynamoDB query items found: " + JSON.stringify ( success ) )
result=success
} )
.catch ( error => console.error ( "DynamoDB query error: " + error ) )
return result
}
dynamoDBUpdate=async function( params )
{
let result=false
await this.dbclient.update ( params )
.promise ()
.then ( success =>
{
console.log ( "DynamoDB update success: " + JSON.stringify ( success ) )
result=true
} )
.catch ( error => console.error ( "DynamoDB update error: " + error ) )
return result
}
generateUpdateExpression=function( data )
{
let updateExpression="SET "
let expressionAttributeValues={}
for ( let attribute in data )
{
if ( attribute.includes ( "sk" ) || attribute.includes ( "pk" ) )
{
continue
}
updateExpression+=`${ attribute }=:${ attribute },`
expressionAttributeValues[ `:${ attribute }` ]=data[ attribute ]
}
updateExpression=updateExpression.slice ( 0, -1 )
return {
updateExpression: updateExpression,
expressionAttributeValues: expressionAttributeValues
}
}
The code should be straightforward enough to understand, so you can adapt it to your needs even if you aren't using DynamoDB for relational purposes. The code adds helpers for adding, deleting, getting, and updating items in DynamoDB. By adding the lambda layer to a lambda function, you get a reusable easy way to access your DynamoDB.
We also add a simple generator function, that takes a payload JSON and builds the DynamoDB update string automatically.
Another common use case is functionality related to authentication tokens. Adding functions for JWT tokens means whenever we need to work with tokens in our code, we can just call the function on our API, and the results will match throughout. First, here's an example for creating a token.
createToken=function( payload )
{
console.log ( "Creating token with payload: " + payload )
return jwt.sign ( {
email: payload.email,
role: payload.userRole,
team: payload.team,
}, secretKey, {
expiresIn: 60 * 15
} )
}
Do note that you don't necessarily need to use the exact same settings as you might want a different expiration time, and you might not have teams or roles in your application. The reason we included them is simple.
Remember the constructor, where we added the roleMap? We can go even further with our API, mixing JWT and the DynamoDB functionality to have our lambdas check if the invoking user is authorized to do what they want to do!
checkAccess=async function( token, requiredLevel )
{
let api=this
let requester=null
let userPayload=null
jwt.verify ( token, secretKey, function( error, payload )
{
if ( !error )
{
console.log ( "Valid token." )
userPayload=payload
}
else
{
console.error ( "Invalid token. Access denied." )
}
} )
if ( !userPayload )
{
return requester
}
await api.dynamoDBGetItem ( "AppTable", `USER-${ userPayload.email }` )
.then ( result =>
{
if ( result )
{
let level=api.roleMap.get ( requiredLevel )
let userLevel=api.roleMap.get ( userPayload.role )
console.log ( "Required level: " + level + ", user level: " + userLevel )
if ( userLevel <= level )
{
console.log ( "User " + userPayload.email + " is allowed access to function." )
requester=userPayload.email
}
else
{
console.warn ( "User " + userPayload.email +
" does not have access to function but tried to access it." )
}
}
else
{
console.error ( "Failed validating user attached to token" )
}
} )
.catch ( error =>
{
console.error ( "Failed validating user attached to token: " + error )
} )
return requester
}
Pretty nifty.
Another common thing lambdas do is they provide responses. So how about we add a simple function, that generates a basic response? Again, customize the settings to your typical use-case.
generateStandardResponse=function()
{
return {
headers: {
"Access-Control-Allow-Headers": "Content-Type",
"Access-Control-Allow-Methods": "DELETE, GET, OPTIONS, POST, PUT",
"Access-Control-Allow-Origin": "*"
},
statusCode: 200
}
}
Now that our layer class has some functionality inside it, let's see an example lambda that uses functions from the layer.
Example usage
Let's create a simple login lambda that takes advantage of the layer. First, we have to include our class from the layer.
const AppAPI=require ( '/opt/nodejs/api' )
Next we create an instance of our app, and add some logging.
'use strict' console.log ( 'Loading login function' ) const api=new AppAPI ()
We continue to create the lambda handler. One thing to note about any login handlers you create: be wary of logging in such lambdas. HTTPS is often relied on to handle hiding a plain text password in transit. This means any logging you do with the password, or any json containing the password, will leak the password in plain text to your logs!
exports.handler=async function( event, context )
{
let response=api.generateStandardResponse ()
let login=JSON.parse ( event.body ) // Expected payload: user, password
await api.dynamoDBGetRelations ( 'AppTable', `USER-${ login.user }`, "USER" )
.then ( result =>
{
if ( result != null )
{
console.log ( "User found: " + login.user )
if ( result.Items.length > 1 )
{
console.error ( "Error: multiple users found! Trying logging in to the first occurrence." )
}
let userItem=result.Items[ 0 ]
let correctPassword=api.authenticate ( login.password, userItem.password )
if ( correctPassword )
{
console.log ( "Correct password for user: " + login.user )
let token=api.createToken ( userItem )
let responseBody={
data: {
token: token
}
}
console.log ( "User " + login.user + " received token " + token )
response.body=JSON.stringify ( responseBody )
}
else
{
console.warn ( "Incorrect password for user: " + login.user )
response.statusCode=403
}
}
else
{
console.warn ( "Unknown user tried to login." )
response.statusCode=403
}
} )
.catch ( error =>
{
console.error ( error )
response.statusCode=500
} )
console.log ( "Response: " + response )
return response
}
As you can see, we take full advantage of the functionality provided by the layer.
Final notes
We have now gotten you started on AWS Lambda layers. There are multiple ways you can improve the starter setup. You can make the classes more granular, for example you could split the api.js into dynamodb-helper.js, authentication.js, and so on, and then only include the relevant bits in your lambdas.
You could also expand all the functionality to better fit and match your use cases. Maybe you want to generate IDs for your database, so you could add a generateID function. Whatever it is that you want to do, I hope this article has helped you on seeing how you can use AWS Lambda Layers with your lambda functions to easily provide common functionality while also making your actual lambdas smaller.