Saleem.dev
Published on

How to Upload Files to AWS S3 Using NodeJs using Serverless Framework

Authors
Table of Contents

Uploading files is a pretty common task for web applications. In this article, I'll share with you the simplest way to upload files (images/videos/PDF..) to AWS S3 using Serverless framework and let you focus on your business logic.

  • In a server-based method, the process follows this flow:

Your user uploads a file to your backend server, your backend server "saves" a temporary space for processing that file, the backend then will transfer the file to any storage service such as S3.

  • In the serverless approach, there are different methods to upload files to S3.

In this article, we'll use S3 directly to upload files which helps to avoid proxying the upload requests to custom servers. This method can significantly reduce network traffic and eventually reduce your bill since S3 is free for data in.

Overview of uploading to S3 using Serverless framework

When we enable direct upload to S3, we need such a mechanism to restrict data input to our S3 bucket. To achieve that, we'll handle this process in two separate requests:

  1. Get signed URL from a custom lambda function.
  2. Use the generated signed URL to upload files directly to S3 via HTTP PUT method.

Create a serverless function

  1. We'll use a serverless component to create an express application on AWS powered by Lambda.

If you want to use pure NodeJs Serverless function you can use Serverless functions instead of components.

npx serverless init express-starter

The generated serverless.yaml looks like this:

# serverless.yaml
app: express-starter
component: express
name: express-starter

inputs:
  src: ./
  1. Update your AWS crudentials inside .env
AWS_ACCESS_KEY_ID=XXX
AWS_SECRET_ACCESS_KEY=XXX

Get S3 signed URL

Write the logic for getting the signed URL from S3 using aws-sdk

To install aws-sdk run the following command: npm i aws-sdk

In this example, I'll assume the files will be uploaded are mp4 video files but feel free to change the logic to make it even dynamic or based on your application logic.

'use strict';

const express = require('express');
const AWS = require('aws-sdk')

const app = express();

app.get('/signed-storage-url', async (_, response) => {
  const s3 = new AWS.S3({
    region: 'eu-west-1'
  })
  const randomID = parseInt(Math.random() * 10000000)
  const Key = `${randomID}.mp4`
  const Bucket = 'your-bucket-name';
  const Expires = 300; // 5 mins

  // Get signed URL from S3
  s3.getSignedUrl('putObject', {
    Bucket,
    Key,
    Expires,
    ContentType: 'video/mp4', // or make it dynamic
    ACL: 'public-read'
  }, (err, url) => {
    response.json({uploadURL: url, Key})
  })
});

// Error handler
app.use((err, req, res) => {
  console.error(err);
  res.status(500).send('Internal Serverless Error');
});

module.exports = app;

Note: make sure you enable the CORS on your S3 bucket, see example below:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3600
    }
]

Upload files from your Front-End

In this example, I use pure JavaScript & HTML to upload files using the presigned url we generated over the backend lambda function above. Addionally, I used TailwindCSS and Plyr to beautify the demo a little bit 🤩.

<!DOCTYPE html>
<html>
    <head>
        <title>Upload file to S3</title>
        <link href="https://unpkg.com/tailwindcss@^2/dist/tailwind.min.css" rel="stylesheet">
        <script src="https://cdn.plyr.io/3.6.3/plyr.js"></script>
        <link rel="stylesheet" href="https://cdn.plyr.io/3.6.3/plyr.css" />
    </head>

    <body>
        <div id="form-upload" class="min-h-screen flex items-center justify-center bg-gray-100">
            <div class="max-w-md w-full py-12 px-6 flex items-center flex-col">
                <label class="w-64 flex flex-col items-center px-4 py-6 bg-white text-blue-500 rounded-lg shadow-lg tracking-wide uppercase border border-blue cursor-pointer hover:bg-blue-600 hover:text-white">
                    <svg class="w-8 h-8" fill="currentColor" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20">
                        <path d="M16.88 9.1A4 4 0 0 1 16 17H5a5 5 0 0 1-1-9.9V7a3 3 0 0 1 4.52-2.59A4.98 4.98 0 0 1 17 8c0 .38-.04.74-.12 1.1zM11 11h3l-4-4-4 4h3v3h2v-3z" />
                    </svg>
                    <span class="mt-2 text-base leading-normal">Select a video</span>
                    <input type='file' id="fileInput" onchange="readVideo()" class="hidden" />
                </label>
            </div>
        </div>

        <div class="hidden min-h-screen flex items-center justify-center bg-gray-100" id="form-uploading">
            <div class="max-w-md w-full py-12 px-6 flex items-center flex-col">
                <label class="w-64 flex flex-col items-center px-4 py-6 bg-white text-blue-500 rounded-lg shadow-lg tracking-wide uppercase border border-blue-500">
                    <svg class="w-8 h-8" fill="currentColor" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20">
                        <path d="M16.88 9.1A4 4 0 0 1 16 17H5a5 5 0 0 1-1-9.9V7a3 3 0 0 1 4.52-2.59A4.98 4.98 0 0 1 17 8c0 .38-.04.74-.12 1.1zM11 11h3l-4-4-4 4h3v3h2v-3z" />
                    </svg>
                    <span class="mt-2 text-base leading-normal">Uploading..</span>
                </label>
            </div>
        </div>

        <div class="hidden min-h-screen flex items-center justify-center bg-gray-100" id="success-form">
            <div class="w-1/2 mx-auto">
                <video id="player" controls></video>
            </div>
        </div>

        <script>
            const API_ENDPOINT = 'https://8jn57o23p4.execute-api.us-east-1.amazonaws.com/signed-storage-url';

            function readVideo() {
                let fileInput = document.getElementById('fileInput');
                if (!fileInput.files.length) return;

                // read the video file
                let reader = new FileReader()
                let videoFile;
                reader.onload = (e) => {
                console.log('length: ', e.target.result.includes('data:video/mp4'))
                if (!e.target.result.includes('data:video/mp4')) {
                    return alert('Wrong file type - MP4 only.')
                }
                uploadVideo(e.target.result)
                }
                reader.readAsDataURL(fileInput.files[0])
            }

            function uploadVideo(video) {
                document.getElementById('form-upload').classList.add('hidden');
                document.getElementById('form-uploading').classList.remove('hidden');

                fetch(API_ENDPOINT, { method: 'GET' })
                    .then(response => response.json())
                    .then(response => {
                        let binary = atob(video.split(',')[1])
                        let array = []
                        for (var i = 0; i < binary.length; i++) {
                            array.push(binary.charCodeAt(i))
                        }
                        let blobData = new Blob([new Uint8Array(array)], {type: 'video/mp4'})
                        let objectUrl = response.uploadURL.split('?')[0];
                        fetch(response.uploadURL, { method: 'PUT', body: blobData})
                            .then(response => {
                                const player = new Plyr('#player', {});
                                player.source = {
                                    type: 'video',
                                    title: 'Example title',
                                    sources: [
                                        {
                                        src: objectUrl,
                                        type: 'video/mp4',
                                        size: 720,
                                        },
                                    ],
                                };
                                document.getElementById('success-form').classList.remove('hidden');
                                document.getElementById('form-uploading').classList.add('hidden');
                            });
                    })
            }
        </script>
    </body>
</html>

Final result

Setting up Serverless function to work with S3 files upload is a straightforward process, however, to avoid any issue, just make sure you setup the bucket's CORS settings correctly.

Edit on GitHub