Fork me on GitHub

Joslyn Esser

MongoDB Sharding Guide: Server Setup on Ubuntu + EC2

Now that we have the overview of sharding out of the way, it’s time to build the servers. I’ll be using EC2 and Ubuntu 9.10 in this guide. If you’d like to skip the manual labor in this part of the guide, the complete build scripts are available on github.

Shard Servers

First, we’ll need to get our shards up and running. If you are deploying to EC2, make sure you have the following completed before you get started:

  1. Create two EBS boot EC2 instances using an up to date Ubuntu 9.10 AMI (as of this writing: ami-7d43ae14)
  2. Create and attach two EBS volumes to /dev/sdf for storing your data on each of the shards
  3. Change your security group to allow for the MongoDB ports we’ll be using (27017-27019)

Once your instances are up and running, ssh in to your first shard server (Shard A) and perform the following:

Add the 10gen MongoDB repository to /etc/apt/sources.list:

deb http://downloads.mongodb.org/distros/ubuntu 9.10 10gen

Update and install MongoDB:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
sudo aptitude update
sudo aptitude safe-upgrade -y
sudo aptitude install mongodb-stable -y

Install XFS support and mount your data drive. XFS is recommended by the 10gen team for performance reasons:

sudo aptitude install xfsprogs -y
sudo mkfs.xfs /dev/sdf
sudo mkdir /mnt/data
sudo mount /dev/sdf /mnt/data

Let’s go ahead and stop MongoDB to change the default configuration:

sudo stop mongodb

Change the default configuration to run MongoDB as a shard server. Change /etc/init/mongodb.conf to look like the following:

description "MongoDB Shard Server"

pre-start script
    mkdir -p /mnt/data/db/mongodb/
    chown -R mongodb:mongodb /mnt/data
end script

start on startup
stop on shutdown

exec start-stop-daemon --start --quiet --chuid mongodb --exec  /usr/bin/mongod -- --shardsvr --dbpath /mnt/data/db/mongodb --logpath /mnt/data/db/mongodb.log --logappend

To make sure our mongo process stays up and running, let’s install and configure monit to startup automatically:

sudo aptitude install monit;
sudo sed -i '/startup/ c startup=1' /etc/default/monit;

Now change the default monit configuration located at /etc/monit/monitrc to watch mongo:

set daemon 60
set logfile /var/log/monit.log
set httpd port 2812 and
  allow admin:monit

check process mongodb with pidfile /mnt/data/db/mongodb/mongod.lock
  start program "/sbin/start mongodb"
  stop program "/sbin/stop mongodb"
  if failed port 27018 then restart
  if 5 restarts within 5 cycles then timeout

Make sure your mounted data drive survives a reboot and is listed in fstab. For performance reasons, we remove the access time attribute:

sudo sed -i '$ a /dev/sdf  /mnt/data  xfs  noatime  0  0' /etc/fstab

Perform a reboot and verify that mongo is running and monitored with monit:

sudo reboot -h now
sudo monit status

Now that Shard A is finished, proceed to Shard B and repeat the above steps. Once finished with both shards, move on to the configuration servers…

Config Servers

Now we’ll need to get our configuration servers up and running. If you are deploying to EC2, make sure you have the following completed before you get started:

  1. Create three micro EC2 instances using an up to date Ubuntu 9.10 AMI (as of this writing: ami-7d43ae14)

Once your instances are up and running, ssh in to your first config server (Config A) and perform the following:

Add the 10gen MongoDB repository to /etc/apt/sources.list:

deb http://downloads.mongodb.org/distros/ubuntu 9.10 10gen

Update and install MongoDB:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
sudo aptitude update
sudo aptitude safe-upgrade -y
sudo aptitude install mongodb-stable -y

Let’s go ahead and stop MongoDB to change the default configuration:

sudo stop mongodb

Change the default configuration to run MongoDB as a config server. Change /etc/init/mongodb.conf to look like the following:

description "MongoDB Config Server"

pre-start script
    mkdir -p /data/db/mongodb/
    chown -R mongodb:mongodb /data
end script

start on startup
stop on shutdown

exec start-stop-daemon --start --quiet --chuid mongodb --exec  /usr/bin/mongod -- --configsvr --dbpath /data/db/mongodb --logpath /data/db/mongodb.log --logappend

To make sure our mongo process stays up and running, let’s install and configure monit to startup automatically:

sudo aptitude install monit;
sudo sed -i '/startup/ c startup=1' /etc/default/monit;

Now change the default monit configuration located at /etc/monit/monitrc to watch mongo:

set daemon 60
set logfile /var/log/monit.log
set httpd port 2812 and
  allow admin:monit

check process mongodb with pidfile /data/db/mongodb/mongod.lock
  start program "/sbin/start mongodb"
  stop program "/sbin/stop mongodb"
  if failed port 27019 then restart
  if 5 restarts within 5 cycles then timeout

Perform a reboot and verify that mongo is running and monitored with monit:

sudo reboot -h now
sudo monit status

Now that Config A is finished, proceed to Config B and Config C and repeat the above steps. Once finished with all three configuration servers, move on to the routing server…

Routing Server

The final server we need is the routing server. If you are deploying to EC2, make sure you have the following completed before you get started:

  1. Create a micro EC2 instance using an up to date Ubuntu 9.10 AMI (as of this writing: ami-7d43ae14)

Once your instance is up and running, ssh in and perform the following:

Add the 10gen MongoDB repository to /etc/apt/sources.list:

deb http://downloads.mongodb.org/distros/ubuntu 9.10 10gen

Update and install MongoDB:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
sudo aptitude update
sudo aptitude safe-upgrade -y
sudo aptitude install mongodb-stable -y

Let’s go ahead and stop MongoDB to change the default configuration:

sudo stop mongodb

Change the default configuration located at /etc/init/mongodb.conf to run MongoDB as a routing server. Change the three config server hostnames (configA,configB,configC) to point to your config servers (If using EC2, make sure to use the private IP addresses to avoid bandwidth charges)

description "Mongos Routing Server"

pre-start script
    mkdir -p /data/db/mongodb/
    chown -R mongodb:mongodb /data
end script

start on startup
stop on shutdown

exec start-stop-daemon --start --quiet --chuid mongodb --exec  /usr/bin/mongos -- --configdb configA,configB,configC --logpath /data/db/mongodb.log --logappend --pidfilepath /data/db/mongodb/mongos.lock

To make sure our routing process stays up and running, let’s install and configure monit to startup automatically:

sudo aptitude install monit;
sudo sed -i '/startup/ c startup=1' /etc/default/monit;

Now change the default monit configuration located at /etc/monit/monitrc to watch mongos:

set daemon 60
set logfile /var/log/monit.log
set httpd port 2812 and
  allow admin:monit

check process mongodb with pidfile /data/db/mongodb/mongos.lock
  start program "/sbin/start mongodb"
  stop program "/sbin/stop mongodb"
  if failed port 27017 then restart
  if 5 restarts within 5 cycles then timeout

Perform a reboot and verify that mongos is running and monitored with monit:

sudo reboot -h now
sudo monit status

Take a deep breath

The systems are now in place! Once you are ready, proceed to configure the cluster and shard your first database.