Software Development

Github

AWS

Automation

Programming

Creating a CI/CD Pipeline using Github and AWS EC2

Hey everyone, today I want to do a tutorial of one of my favorite ways to deploy a quick REST API, using NodeJS / Express, to an Amazon EC2 server. This technique essentially enables your EC2 server to respond to changes pushed to your Github repository.

First things first, “Hello World”

Create a new directory on your machine for a NodeJS / Express app to live. I’m calling mine rekognition

Open up a terminal and run this command to initialize a package.json npm init -y

Install express.js npm i express

Create a basic express server with a ‘hello world’ at the default route

const express = require('express');
const app = express();
const port = 3000;

app.get('/', (req, res) => {
    res.send('Hello World!');
});

app.listen(port, () => {
    console.log(`Example app listening at http://localhost:${port}`);
});

Test to make sure your are seeing ‘Hello World!’ at http://localhost:3000/

Good job, step one complete! 💪

Next let’s push our code to Github

Initialize a Git repo in your directory and make your first commit —

git init
git add .
git commit -m "First commit"

Create a remote repository on your Github account

Add the new repo as the origin and push your branch

git remote add origin <REMOTE_REPOSITORY_URL>
git push -u origin master

Replace <REMOTE_REPOSITORY_URL> with the link to your remote repo that you just created.

You might need to change your repository settings to use ‘master’ as the default branch instead of ‘main’

After you do this you can delete the ‘main’ branch.

Step 2: complete! We’re cooking with gas ⛽️

Let’s go to the cloud — setup the Ec2 server

Here are the steps to launch your EC2 Instance:

• Log in to your AWS Management Console.
• Navigate to the EC2 dashboard and launch a new instance.

• Choose an appropriate AMI (Amazon Machine Image), I am using Amazon Linux 2.
• Select an instance type (e.g., t2.micro if you are under the free tier).
• Configure instance details, add storage, and add tags as necessary.
• Configure a security group to allow inbound traffic on ports you’ll need (at least TCP port 22 for SSH and your application’s port, and then 3000 for web traffic).
• Review and launch the instance, ensure that you save the key pair used for SSH access.

Now let’s test logging into your server via SSH on a terminal:

ssh ec2-user@<YOUR_SERVER_IP> -i <YOUR_PRIVATE_KEY>.pem

Replace the variables there with the values you defined for your EC2 instance.

If you see this you’ve connected to your server, congrats! 👏

We’re not done just yet though — we’re just getting started 😏

Let’s install NodeJS on our server:
sudo yum install -y nodejs

Install pm2 globally on our server via npm

sudo npm i -g pm2

Install git on the server and configure your credentials:

sudo yum install git -y

git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"

Okay now we’re going to create a var/www/ folder if it does not already exist:

sudo mkdir -p /var/www

Then add ownership to our ec2-user:

sudo chown -R ec2-user:ec2-user /var/www

Navigate to the directory from the root of your server — not the /home folder people 👌 cd ~/var/www

Now from here we will clone our git repository: git clone https://github.com/USERNAME/REPO_NAME.git YOUR_PROJECT_DIRECTORY

Replace [USERNAME] [REPO_NAME] and [YOUR_PROJECT_DIRECTORY] with your own values.

Next we change directory into our repository, so cd YOUR_PROJECT_DIRECTORYcand reate the ecosystem.config.js file for our pm2 setup:

nano ecosystem.config.js

It looks like this:

module.exports = {
  apps : [{
    name: "my-app",  // A name for your application
    script: "app.js",  // The script file to launch the app
    instances: "max",  // This can be set to the number of CPU cores to use, or "max" to use all available
    autorestart: true,  // Automatically restart app if it crashes
    watch: false,  // Enable/disable watching file changes for automatic restart
    max_memory_restart: '1G',  // Optional: Restart app if it reaches a memory limit
    env: {
      NODE_ENV: "development",
    },
    env_production: {
      NODE_ENV: "production",
    }
  }]
};

Coolbeans, okay — once that is created and saved, let’s start up the app with pm2:

pm2 start ecosystem.config.js

Now we can check that we have a ‘Hello World!’ from our server on port 3000:

To make pm2 start automatically on server restarts let’s run:

pm2 save
pm2 startup

To check your server is running you can do pm2 status , you can read the docs on pm2 here

If you’ve made it this far you are a superstar ⭐️

Integration with Github Actions

This is the last step in our process. We’re going to create a .yml file for Github actions to update our server anytime a change is pushed to our master branch

In your repository, go to the ‘Actions’ tab and create a new custom workflow. I am calling mine nodejs.yml

Here is the .yml file for updating the repo and restarting pm2:

name: Node.js CI/CD on Amazon Linux

on:
  push:
    branches: [ master ]

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Install dependencies
      run: npm install

    - name: Deploy to AWS EC2
      uses: appleboy/scp-action@master
      with:
        host: ${{ secrets.REMOTE_HOST }}
        username: ${{ secrets.REMOTE_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        source: "./"  # This now correctly points to the current directory
        target: "/var/www/your_project_directory"

    - name: Restart PM2 application
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.REMOTE_HOST }}
        username: ${{ secrets.REMOTE_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          cd /var/www/your_project_directory
          pm2 reload all  # This command restarts your application

Make sure to update ‘your_project_directory’

Lastly, you’ll need to add these environment secrets REMOTE_HOSTthis is the ip address of your server, REMOTE_USER this is ec2-user for Amazon Linux 2 AMI, and then paste the contents of your .pem file as the SSH_PRIVATE_KEY You can do all of this in the repository settings:

That’s it! We’re done yay 😁

You can test this all by changing the ‘Hello World’ message and push that change to your master branch, then you should see the update at http://your.ip.address:3000

It is a bit of a process to set up, but when you’re developing it’s really nice to have an easy CI/CD pipeline automatically deploying changes to your REST API server. That way you can quickly deploy the changes live and see them in the frontend of your web / mobile application.

Let me know if this was useful in the comments and thanks for reading!!

Peace out,

— Wes