Automated deployment of node.js apps
This article has a simple aim to improve the process of developing and deploying node.js web applications. The Continuous Integration folks will refer to what I am covering as an aptly named Automated Deployment. As you explore this guide you will learn:
- How to run a node.js script using Upstart.
- How to manage Jenkins (was Hudson) with Git.
- How to make and use Makefiles.
Lets crack on…
I have detailed an environment scheme below which works well for me. You are free to assign your machine names what you please but following this guide will be easiest if you follow my conventions.
Hostname | Description | Operating System |
---|---|---|
localhost | Your machine you develop from | Mac / Windows / Linux |
ci.app | Jenkins and Git Server | Ubuntu |
int.app | node.js application server | Ubuntu |
Getting a working node.js app and environment
You will need to have an environment to run your app in. To save on resources I prefer using virtual machines hosted on my localhost. This also means you are not tied to a network in order to develop (great for coding on the train/in a coffee shop).
You can run node.js apps on your localhost but for this article we are just focusing the deployment process to an integration server (int.app).
The correct development flow for a node.js app should be that you test on localhost, then checkin when you are happy with your changes. The checkin will then trigger a build and automatically get built to int.app. We will be in this sweet spot by the end of this article.
Setting up your virtual machines
It is fairly straightforward (and free) to get a Linux virtual machine up and running these days. Following the steps on https://help.ubuntu.com/community/VirtualBox should be a good starting point.
I suggest you set up one machine (int.app) first with a barebones configuration and clone it to create ci.app, changing the hostname (sudo hostname newhostname
) and getting a new IP address sudo dhclient
on the cloned machine.
Configuring int.app
Our initial development centric dependencies are Git, node.js, Express and NPM.
Login to int.app
ssh int.app
Install Git
apt-get install git git-core
Install node.js
cd /tmp/ git clone https://github.com/joyent/node.git cd node ./configure make make install
Install NPM
apt-get install curl curl http://npmjs.org/install.sh | sudo sh
Install Express
npm install express
Now you have everything you need to get a node.js app up and running. Let’s create our demo app.
mkdir -p /data/apps/hello_world
Add these contents to /data/apps/hello_world/app.js
.
var express = require('express') , app = express.createServer() app.get('/', function(req, res){ res.send([ '<h1>Hello World</h1>' ].join('\n')); }); app.listen(3000); console.log('Express app started on port 3000');
Run your node.js app
node /data/apps/hello_world/app.js
Getting and using the IP address of int.app
SSH into int.app and run:
ifconfig -a
- Grab the IP address from ‘inet addr’. This should look something like:
192.188.0.23
. - Add this to your
/etc/hosts
file on your localhost like:192.188.0.23 int.app
Testing the app
Now let’s test if everything is running ok. Open up http://int.app:3000/. With any luck you should be seeing the text “Hello World”.
Getting a build process working
We will be using Git as our source control management tool (VCS). Have a read of Joel Spolsky’s distributed VCS post to understand why we are using Git over something like SVN.
Setting up a Git server on ci.app
Git works in a de-centralised way but we need to have a central Git repository that we and other developers can push their changes to and other tools like Jenkins can pull updates from.
On ci.app run:
useradd git su git sudo mkdir -p /data/git/hello_world.git sudo chown git -R /data/git/hello_world.git cd /data/git/hello_world.git git --bare init exit
On localhost you can now run:
git clone ssh://ci.app/data/git/hello_world.git ~/hello_world cd hello_world git remote add origin ssh://git@ci.app/data/git/hello_world.git
Now you have an empty directory on your local machine where you can develop from. Let’s add app.js
we made earlier to this repository.
Add these contents to ~/hello_world/app.js
var express = require('express') , app = express.createServer() app.get('/', function(req, res){ res.send([ '<h1>Hello World</h1>' ].join('\n')); }); app.listen(3000); console.log('Express app started on port 3000');
Now lets check it in.
git add --all git commit -m 'Initial checkin' app.js
Let’s push it to our remote Git server
git push origin master
Moving the app automatically
We are going to use make as a tool to bundle commands. We will use Make to move the application from the Git repository to the /data/apps/hello_world
directory.
On localhost, add these contents to ~/hello_world/Makefile
install : sudo mkdir -p /data/apps/hello_world sudo cp ./app.js /data/apps/hello_world/
Check this file in and push changes:
git add --all git commit -m 'adding build steps' git push origin master
Now you could run this command and have the app put in the correct location for you:
make install
Automating the build
Jenkins is a very useful tool in the process of Continuous Integration. At the most basic level Jenkins is a tool to allow you to schedule things to happen based on timings or events such as a code checkin.
Installing Jenkins
Let’s install Jenkins on ci.app.
apt-get install jenkins
Once installed you should be able to load Jenkins in your browser: at http://ci.app:8080/.
Adding a new job to Jenkins
Here you can add a ‘job’ by:
- Selecting ‘New Job‘.
- Call the Job name “Hello World”.
- Select the option ‘Build a free-style software project’.
- Save
Once created you will be met with a page with a bunch of inputs. We will come back to that in a moment but first we need to add Git support to Jenkins.
Adding Git support to Jenkins
- Go to http://ci.app:8080/pluginManager/available.
- Click on the enable Git Plugin checkbox.
- Click install at the bottom of the page.
- Return back to your job configuration page (http://ci.app:8080/job/Hello World/configure)
- Select the Git option under the Source Code Management section.
- Add this url into the url field:
ssh://git@localhost/data/git/hello_world.git
- Select the Poll SCM checkbox under Build Triggers and enter:
* * * * *
(This will make Jenkins check every minute for changes to source files.) - Save the configuration
- Click ‘build now’ to check everything is working ok.
As it stands, the Jenkins job is not being all that useful but we will discover it’s power later on.
Deploying your app using Upstart
Node.js scripts need to be run as daemons for hosting websites, which can make the deployment process a bit fiddly. We can use tools like Upstart to run and manage node.js scripts.
Install Upstart
apt-get install upstart
Adding Upstart app configuration
On localhost, add these contents to ~/hello_world/hello_world.conf
#!upstart description "node.js hello world app server" author "James Broad" start on startup stop on shutdown script export HOME="/root" exec /usr/local/bin/node /data/apps/hello_world/app.js 2>&1 >> /var/log/node.log end script
We will be putting this file into /etc/event.d/hello_world
on int.app later on. Once in that location you can run sudo start hello_world
Continuously deploying your app
Now that Jenkins is set up to keep an eye on our files, we can issue it some commands to deploy and run the app.js application.
Let’s return to localhost and add a deployment step to Makefile. This will copy the files needed to install and run our application.
Add these contents to ~/hello_world/Makefile
app_dir = /data/apps/hello_world temp_install_dir = /tmp/hello_world deployment_hostname = int.app install : sudo mkdir -p $(app_dir) sudo cp ./app.js $(app_dir)/app.js sudo cp ./hello_world.conf /etc/event.d/hello_world deploy : # Get rid of old temp installs ssh $(deployment_hostname) sudo rm -rf $(temp_install_dir) # Copy files over to remote machine rsync -r . $(deployment_hostname):$(temp_install_dir) # Install our app to the right location ssh $(deployment_hostname) cd $(temp_install_dir)\; make install ssh $(deployment_hostname) cd $(temp_install_dir)\; make start_app start_app : sudo start --no-wait -q hello_world
Check in and push your changes.
git add --all git commit -m 'adding deployment steps' git push origin master
Handling environment permissions
On int.app you will NEED to set up your SSH Keys to allow passwordless ssh access and add paswordless sudo rights for the user jenkins so it can carry out tasks that require sudo such as service restarting.
Passwordless SSH access
To set up passwordless SSH access for Jenkins, log into ci.app as the user jenkins ssh jenkins@ci.app
. Run and complete the prompts:
ssh-keygen
Copy the output of ~/.ssh/id_dsa.pub
or ~/.ssh/id_rsa.pub
(depending on what option you chose) to the file ~/.ssh/authorized_keys
on int.app.
Now SSH to int.app and add the jenkins user adduser jenkins
, this will be needed for when Jenkins is installing stuff onto int.app.
Now we need to log into ci.app as user jenkins to be able to accept the SSH Keys:
ssh jenkins@ci.app ssh jenkins@int.app exit exit
Passwordless sudo rights
For enabling passwordless sudo rights to the jenkins user you will need to run: sudo visudo
and append the file with the contents:
# Allow Jenkins to run certain commands as sudo without password jenkins ALL = NOPASSWD: /bin/mkdir, /bin/rm, /bin/cp, /sbin/start, /sbin/stop
Adding shell script stages to Jenkins
Go to your hello world configuration panel (http://ci.app:8080/job/Hello World) and choose ‘add build step’ under the Build section.
- Select ‘Execute shell’ and add this line:
make deploy
. - Save the settings.
- Run a build.
If everything went according to plan you should be able to load http://int.app:3000 and see the text ‘Hello World’.
Verifying deployments
Instead of us having to check the app is running every time a build happens, we can get Jenkins to do that for us by checking a url returns 200 response code.
Lets add a new Freestyle job in Jenkins and call it “Hello World Monitor”. Leave everything blank but click the checkbox for Monitor Site and add the url http://int.app:3000/
.
We can now return back to our Hello World job and check the Build other projects checkbox. Then add Hello World Monitor
(it will auto-suggest). Save this configuration and run a build.
Finally
If you have made it this far, thanks for reading, hopefully everything went well?
If everything went well, you are in a good place to start adding more Continuous Integration elements such as unit tests, JSLinting and many other automated steps.
There are a number of things I didn’t go into much depth on such as the node.js app itself, there are plenty of articles out there about how to do this. A good resource worth checking out is http://howtonode.org.
Feel free to get in touch if you need any help but be warned I have a busy day job so I may take time to respond.
Image courtesy of Me
12 Comments
[...] Quelle: http://www.carbonsilk.com/node/deploying-nodejs-apps/ This entry was posted in Uncategorized. Bookmark the permalink. ← mongly.com – MongoDB Tutorial [...]
nice post
Very thorough article! Thanks very much for this. It will help me a lot.
Fantastic article, thank you very much!
[...] The Continuous Integration folks will refer to what I am covering as an aptly named Automated Deployment . As you explore this guide you will learn: Automated deployment of node.js apps – Carbon Silk – Developing ideas by James Broad [...]
[...] Automated deployment of node.js apps – Carbon Silk – Developing ideas by James Broad This article has a simple aim to improve the process of developing and deploying node.js web applications. The Continuous Integration folks will refer to what I am covering as an aptly named Automated Deployment . [...]
It looks like if you want STDERR output to go into /var/log/node.log you should reverse the order of the file handle redirection and the file-append redirection.
exec node app.js >> /var/log/node.log 2>&1
[...] http://www.carbonsilk.com/node/deploying-nodejs-apps [...]
Thank you!
Thank you, Thank you!
Thanks, that was very helpful (y)
Good article, helped me finally flesh out my CI process.
Any reason why you chose to use Make instead of something more in vogue like Ant, Grunt, Capistrano, et al? Not criticizing — I ended up using your skeleton to quite good effect — just curious.
@Ændrew Glad you found it useful!
I chose Make as it is available on most machines out the box and is as simple as it gets with regards to the configuration.