A Step-by-Step Guide to Installing GlusterFS on Ubuntu
In modern computing, the need for scalable, resilient, and highly available storage is more critical than ever. Whether you’re managing large datasets, running virtual machines, or building a fault-tolerant infrastructure, a distributed file system is often the answer. GlusterFS stands out as a powerful, open-source solution that lets you pool storage resources from multiple servers into a single, unified namespace.
Unlike complex storage systems that rely on a central metadata server, GlusterFS uses a distributed, hash-based architecture. This design eliminates single points of failure and allows for linear scaling of both performance and capacity. This guide will walk you through the complete process of setting up a reliable GlusterFS storage cluster on Ubuntu servers.
Prerequisites: What You’ll Need
Before diving into the installation, ensure you have the following setup prepared. For this tutorial, we will assume you are configuring a basic two-node cluster.
- Two or more servers running a fresh installation of Ubuntu (e.g., 18.04, 20.04, or newer).
- Root or sudo access on all servers.
- A dedicated storage device or partition on each server for the GlusterFS brick. While you can use the root partition, a separate disk is highly recommended for production environments.
- Properly configured hostnames and network connectivity. Each server must be able to resolve the others by name. You can achieve this with a local DNS server or by adding entries to the
/etc/hostsfile on each node.
For example, your /etc/hosts file on both servers might look like this:
192.168.1.100 gfs-node1
192.168.1.101 gfs-node2
Step 1: Install GlusterFS Server Software
The first step is to install the necessary GlusterFS packages on all servers that will be part of the storage pool.
Open a terminal on each server and run the following commands to update your package list and install the glusterfs-server package:
sudo apt update
sudo apt install glusterfs-server -y
Once the installation is complete, the GlusterFS daemon, glusterd, should be started and enabled automatically. You can verify its status with this command:
sudo systemctl status glusterd
You should see an “active (running)” status. If not, enable and start it manually:
sudo systemctl enable glusterd
sudo systemctl start glusterd
Step 2: Configure the Firewall
For the nodes to communicate, you must open the necessary ports in your firewall. GlusterFS requires port 24007 for the daemon and additional ports for each brick. A simple rule is to allow all traffic from the other nodes in the cluster.
If you are using UFW (Uncomplicated Firewall), run the following command on gfs-node1 to allow connections from gfs-node2:
sudo ufw allow from 192.168.1.101
Then, run this command on gfs-node2 to allow connections from gfs-node1:
sudo ufw allow from 192.168.1.100
For tighter security, you can open only the specific ports required. The Gluster daemon uses port 24007, and each brick will use a port starting from 24009.
Step 3: Create the Trusted Storage Pool
A trusted storage pool is a collection of servers that will participate in the storage cluster. You only need to run this command from one node.
From gfs-node1, probe the other node (gfs-node2) to add it to the pool:
sudo gluster peer probe gfs-node2
You should see a message confirming that the probe was successful. To verify the status of the pool, run:
sudo gluster peer status
The output will list all nodes in the pool and confirm their connection state. You have now successfully created the foundation of your storage cluster.
Step 4: Prepare the Storage Bricks
A brick is the fundamental unit of storage in GlusterFS, which is simply a directory on a server that will be part of a volume. You must create a brick directory on each node.
On both gfs-node1 and gfs-node2, create a directory that will serve as the brick. It’s best practice to place this on a dedicated partition.
sudo mkdir -p /gluster/brick1
This creates a directory named brick1 inside a /gluster folder on both servers. This is where the data for your volume will be stored.
Step 5: Create a GlusterFS Volume
With the storage pool and bricks ready, you can now create a GlusterFS volume. A volume is the logical storage unit that clients will mount and use. GlusterFS supports several volume types, but one of the most common for high availability is a replicated volume.
A replicated volume keeps an identical copy of the data on each brick in the volume, ensuring that if one server goes down, the data is still accessible from the other.
From any single node (e.g., gfs-node1), run the following command to create a replicated volume named myvol1:
sudo gluster volume create myvol1 replica 2 transport tcp gfs-node1:/gluster/brick1 gfs-node2:/gluster/brick1
Let’s break down this command:
gluster volume create myvol1: Creates a new volume namedmyvol1.replica 2: Specifies the volume type as replicated, with 2 copies of the data (one on each node). The number must match the number of bricks specified.transport tcp: Defines the network protocol to use.gfs-node1:/gluster/brick1 gfs-node2:/gluster/brick1: Specifies the bricks that will make up this volume.
Step 6: Start the GlusterFS Volume
After creating the volume, you need to start it to make it available for use.
sudo gluster volume start myvol1
You can verify that the volume has started successfully and check its configuration with the info command:
sudo gluster volume info
This will display detailed information about myvol1, including its type, status, and the bricks it contains.
Step 7: Mount the GlusterFS Volume on a Client
Your distributed file system is now running. To use it, you need to mount it on a client machine. The client can be one of the GlusterFS nodes or any other server on the same network.
First, install the GlusterFS client package on the machine you want to mount the volume on:
sudo apt update
sudo apt install glusterfs-client -y
Next, create a directory where you will mount the shared volume:
sudo mkdir /mnt/gluster-storage
Finally, mount the GlusterFS volume using the hostname of any node in the storage pool:
sudo mount -t glusterfs gfs-node1:/myvol1 /mnt/gluster-storage
You can now use the /mnt/gluster-storage directory as a standard file system. Any file you create in this directory will be automatically replicated between gfs-node1 and gfs-node2.
Making the Mount Permanent
To ensure the volume is automatically mounted after a reboot, add an entry to your /etc/fstab file.
sudo nano /etc/fstab
Add the following line to the end of the file:
gfs-node1:/myvol1 /mnt/gluster-storage glusterfs defaults,_netdev 0 0
The _netdev option is crucial, as it tells the system to wait until the network is available before attempting to mount the file system.
Final Thoughts and Security Tips
You have successfully deployed a robust, highly available storage solution with GlusterFS on Ubuntu. This setup provides a powerful foundation for building resilient applications and services.
Security Best Practice: For production environments, it is highly recommended to use a dedicated, private network interface for all GlusterFS traffic. This isolates storage communication from public networks, enhancing both security and performance. By configuring your nodes and firewall to use this private network, you create a more secure and efficient storage cluster.
Source: https://kifarunix.com/install-and-setup-glusterfs-on-ubuntu-18-04/
