Graphite, nginx and supervisord

In case you haven’t met it before, Graphite is a really nice real-time graphing tool. It’s simple to add data into it, and it has a nice web-interface for viewing the graphs. Think Cacti, but 10x easier to use and prettier. Graphite uses a custom data store called Whisper (rather than RRDTool which Cacti is based on) which makes it really easy to just push data into it whenever you want. Installation it is also fairly easy – just follow the (recently much improved) documentation on the Graphite site.

It comes with a development server that you can use to play around with the web interface, although it is stressed in the documentation that it is only a development server, and you really should be using Apache and mod_python. And that’s a bit annoying, because I wanted to use nginx for some other things on my system, so I don’t want Apache stealing port 80.

So, to get round this problem, I decided to install Apache and mod_python, and let Graphite think it has won.

aptitude install apache2 libapache2-mod-python libapache2-mod-wsgi

One of the nice features about nginx is that it works really well as a reverse proxy server, so I bound Apache to bind to 127.0.0.1 only and listen on port 81. I just had to set up the following in Apache’s config, drop the example vhost config file provided by Graphite into /etc/apache2/sites-available/ and symlink it into /etc/apache2/sites-enabled/. The following in /etc/apache2/ports.conf is all it takes to hide Apache from view:

NameVirtualHost *:81
Listen 127.0.0.1:81

I made a few changes to the Graphite vhost file, commenting out the line to configure WSGISocketPrefix (I found it just wasn’t needed) and set a server name that nginx will later use to proxy requests to:

ServerName graphite-apache.localhost

Since this is a new machine, I didn’t yet have nginx installed. I also want to ensure that nginx runs from within supervisord, and installing these is easy:

aptitude install nginx-full supervisor

Out of the box, nginx will run using an init script which I don’t want. After making sure that nginx wasn’t running, I removed the script at /etc/init.d/nginx so it wouldn’t be used any more. To run nginx under supervisord, the only change you need to make is to set it to run in the foreground – i.e. not as a daemon. Edit /etc/nginx/nginx.conf and add the following (somewhere near the top of the file, in the root namespace):

daemon off; # run in the foreground so supervisord can look after it

To tell supervisord about it, create a file called /etc/supervisor/conf.d/nginx.conf containing the following:

[program:nginx]
command=/usr/sbin/nginx

You can then start/stop/restart/etc nginx using supervisorctl, for example:

supervisorctl start nginx

I then setup nginx to act as a reverse proxy server on port 80, and pass requests back to Apache on port 81:

server {
    listen 80;
    server_name graphite.localhost; # change this to the hostname you want to use for graphite

    access_log /srv/Web/graphite.eos/logs/access.log; # change this to where you store access logs
    error_log /srv/Web/graphite.eos/logs/error.log;   # change this to where you store access logs

    location / {
        proxy_pass http://graphite-apache.localhost:81/; # this is where Apache is set to listen
    }
}

To check the config is valid and doesn’t contain any syntax errors just run:

nginx -t

Whenever you make a change to nginx config, or add new config in like above, you need to tell nginx. One way is to just restart it (supervisorctl restart nginx) but nginx also supports the SIGHUP signal to reload the config without needing a restart. A really convenient one-liner for reloading nginx config is:

kill -HUP `cat /var/run/nginx.pid`

Finally, just add the 2 hosts (the hidden Apache one, and the visible nginx one) to /etc/hosts pointing at IP 127.0.0.1, for example:

127.0.0.1    graphite-apache.localhost graphite.localhost

Assuming you’ve got Graphite installed, if you open your web browser and go to http://graphite.localhost/, you should see the Graphite web interface.

This entry was posted on November 20, 2011 at 6:01 pm and is filed under Home Server. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.