Atlantic.Net Blog

How to: DRBD Replication and Configuration

Introduction

This how-to will help walk you through the DRBD replication and configuration process. Distributed Replicated Block Device (DRBD) is a block level replication between two or more nodes and is used as a replacement for shared storage by created a networked mirror. DRBD is used in environments that require systems or data to be Highly Available.

Prerequisites

* Two servers running Debian GNU/Linux Distribution. Other versions of Linux will work as well, but the installation packages may be different.
* Both servers should be directly cross-connected together, or have a separate Network Interface for private communication.
* Both servers should have the same partitioning. This walkthrough assumes that both systems have a single /dev/sdb device that is going to be used as the
DRBD volume.

If you do not have a server or need a new one, check out the industry-leading VPS hosting from Atlantic.Net.

Network:

The first part of the process is to make sure that both nodes can talk to each other. This can be done by configuring both nodes with a static private IP address.

You can modify the network interface file directly. Here is an example of the /etc/network/interfaces file of one of our nodes:

# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto eth1
iface eth1 inet static
        address  10.0.10.10
        netmask  255.255.255.0

auto vmbr0
iface vmbr0 inet static
        address  172.16.10.10
        netmask  255.255.255.0
        gateway  172.16.10.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

In our setup, host01 is configured to use IP 10.0.10.10 and host02 is configured to use IP 10.0.10.11.

After making changes to the /etc/network/interfaces file, restart networking or bring up the new interface and make sure that both servers can communicate
with each other on their new private IP.

Disk for DRBD:

Partitioning

Use parted, where /dev/sdb is the device we want to use:

parted /dev/sdb

Once done, the below commands will create your first partition on /dev/sdb and will be used to make a 100GB volume for our first VM/DRBD device. This partition will be /dev/sdb1.

(parted) mkpart primary 0GB 100GB

It’s important to note that the sizes listed are the disk locations listed in Gigabytes. This tells parted to create a new partition at disk size location 0GB through disk size location 100GB. To add a second partition, your starting disk size location about be 100GB, see below:

(parted) mkpart primary 100GB 200GB

If you want to double-check and review your existing partitions to make sure that you using the right disk size locations, run the following and take a look at the results:

(parted) print all

Number  Start   End     Size    File system  Name     Flags
 1      0GB     100GB   100GB                primary  
 2      100GB   200GB   100GB                primary  
 3      200GB   300GB   100GB                primary  
 4      300GB   400GB   100GB                primary  
 5      400GB   500GB   100GB                primary

DRBD configuration:

Software installation:

Install the DRBD user tools. On ALL DRBD nodes, run:

apt-get update && apt-get install drbd8-utils

 

Prepare DRBD configuration:

Replace /etc/drbd.d/global_common.conf with the following content:

global { usage-count no; }
common {
        syncer { rate 30M; verify-alg md5; }
        handlers { out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root"; }
}

Configuring the rate of synchronization:

A good rule of thumb for this value is to use about 30% of the available replication bandwidth or IO.

Create a resource configuration file:

Create a new file, /etc/drbd.d/r0.res, on ALL DRBD nodes.

resource r1 {
        protocol C;
        startup {
                wfc-timeout  0;     # non-zero wfc-timeout can be dangerous 
                degr-wfc-timeout 60;
                become-primary-on both;
        }
        net {
                cram-hmac-alg sha1;
                shared-secret "my-secret";
                allow-two-primaries;
                after-sb-0pri discard-zero-changes;
                after-sb-1pri discard-secondary;
                after-sb-2pri disconnect;
        }
        on host01 {
                device /dev/drbd1;
                disk /dev/sdb1;
                address 10.0.10.10:8001;
                meta-disk internal;
        }
        on host02 {
                device /dev/drbd1;
                disk /dev/sdb1;
                address 10.0.10.11:8001;
                meta-disk internal;
        }
  disk {
    no-disk-barrier;
    no-disk-flushes;
  }
}

If you start adding addition resources, the following fields will need to be updated in your new resources:

resource r1        --> resource r2
device /dev/drbd1; --> device /dev/drbd2
disk /dev/sdb1;    --> disk /dev/sdb2
address 10.0.10.10:8001; --> address 10.0.10.10:8002
device /dev/drbd1; --> device /dev/drbd2
disk /dev/sdb1;    --> disk /dev/sdb2
address 10.0.10.11:8001; --> address 10.0.10.11:8002

Bring DRBD Online:

On both servers, start DRBD:

/etc/init.d/drbd start

Now create the device metadata, also on both nodes:

drbdadm create-md r1

Bring the device up, also on both nodes:

drbdadm up r1

Now you can check the current status of the new DRBD volume, it should look like this on both nodes:

host01:~# cat /proc/drbd
version: 8.3.13 (api:88/proto:86-96)
GIT-hash: 83ca112086600faacab2f157bc5a9324f7bd7f77 build by [email protected], 2012-10-09 12:47:51
 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:2096348

DRBD has successfully allocated resources and is ready for further configuration. Start the initial synchronization (only on one node!!!):

drbdadm -- --overwrite-data-of-peer primary r1

Wait until the initial sync is finished (depending on the size and speed this process can take some time):

host01:~# watch cat /proc/drbd

Once completed, check to see that your DRBD is starting in Primary/Primary mode. In order to do this, stop DRBD service on both nodes:

/etc/init.d/drbd stop

And start again on both nodes:

/etc/init.d/drbd start

Now DRBD should be in the Primary/Primary mode:

host01:~# cat /proc/drbd
version: 8.3.13 (api:88/proto:86-96)
GIT-hash: 83ca112086600faacab2f157bc5a9324f7bd7f77 build by [email protected], 2012-10-09 12:57:41
 1: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----
    ns:1192004977 nr:0 dw:1191846322 dr:705864868 al:282022 bm:32 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0

At this point, you have completed your DRBD setup, and the DRBD resource can be further configured for use as local storage. Thank you for following along and feel free to check back with us for further updates or check out related articles like Configuring LVM on DRBD in our blog.

VPS hosting is just one of the many hosting services offered by Atlantic.Net – We also offer dedicated, managed and HIPAA compliant hosting solutions. Contact us today for more information on any of our services!

Get started with 12 months of free cloud VPS hosting

Free Tier includes:
G3.2GB Cloud VPS Server Free to Use for One Year
50 GB of Block Storage Free to Use for One Year
50 GB of Snapshots Free to Use for One Year