A complete guide to setting up a distributed, replicated GlusterFS filesystem across multiple Ubuntu 22.04 nodes, including installation, volume creation, client mounting, maintenance, and troubleshooting.
Setting Up GlusterFS on Ubuntu 22.04
GlusterFS is a scalable, distributed filesystem that ensures high availability and fault tolerance. This guide walks through configuring GlusterFS on three Ubuntu 22.04 servers, enabling a replicated filesystem across nodes. We also cover maintenance tasks, performance optimizations, and troubleshooting techniques for production-ready setups.
Overview
We will configure GlusterFS across three servers:
- web: 10.0.0.2
- web-node1: 10.0.0.5
- web-node2: 10.0.0.6
By replicating data across these nodes, GlusterFS ensures fault tolerance and consistent access to data, even if a node fails.
Prerequisites
- Three Ubuntu 22.04 servers with sudo privileges.
- Network connectivity between the servers.
- Update the
/etc/hostsfile on each server with the following entries:
10.0.0.2 web
10.0.0.5 web-node1
10.0.0.6 web-node2
Installation Steps
Step 1: Install GlusterFS on All Nodes
Run the following commands on each server:
sudo apt update
sudo apt install glusterfs-server -y
sudo systemctl start glusterd
sudo systemctl enable glusterd
Step 2: Configure the GlusterFS Trusted Pool
From the primary node (web), probe the other nodes:
sudo gluster peer probe web-node1
sudo gluster peer probe web-node2
Tip: Run gluster peer status on any node to verify that all peers are connected.
Step 3: Create Brick Directories
Create brick directories on all nodes to store GlusterFS data:
sudo mkdir -p /glusterfs/brick1/gv0
Step 4: Create and Start the Volume
On the primary node, create a replicated GlusterFS volume:
sudo gluster volume create gv0 replica 3 web:/glusterfs/brick1/gv0 web-node1:/glusterfs/brick1/gv0 web-node2:/glusterfs/brick1/gv0
sudo gluster volume start gv0
Step 5: Mount the GlusterFS Volume
Install the GlusterFS client and mount the volume on each node:
sudo apt-get install glusterfs-client -y
sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs web:/gv0 /mnt/glusterfs
To auto-mount at boot, add the following line to /etc/fstab:
web:/gv0 /mnt/glusterfs glusterfs defaults,_netdev 0 0
Testing the Setup
Create a test file on one node to verify data replication:
sudo touch /mnt/glusterfs/testfile.txt
Check the other nodes -- the file should appear on all of them.
Adding a New Node
Step 1: Prepare the New Node
Install GlusterFS and add its IP to /etc/hosts on all servers:
10.0.0.7 web-node3
Step 2: Add the Node to the Trusted Pool
sudo gluster peer probe web-node3
sudo gluster peer status
Network Optimization
Enhance performance by tuning TCP settings:
sudo sysctl -w net.core.rmem_max=16777216
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_rmem='4096 87380 16777216'
sudo sysctl -w net.ipv4.tcp_wmem='4096 65536 16777216'
sudo sysctl -w net.core.netdev_max_backlog=250000
GlusterFS Maintenance
Rebalancing Volumes
Rebalance the volume after adding or removing bricks:
sudo gluster volume rebalance gv0 start
Checking Volume Health
Monitor volume health and heal inconsistencies:
sudo gluster volume heal gv0 info
sudo gluster volume heal gv0
Troubleshooting
Peer Not Connected
If a peer shows as disconnected, verify:
- Network connectivity between nodes.
- Ports 24007-24009 are open.
- The
glusterdservice is running on all nodes.
sudo systemctl status glusterd
By following this guide, we have set up a distributed and replicated GlusterFS filesystem across multiple nodes. This configuration ensures high availability and fault tolerance, making it suitable for demanding environments. We recommend regularly monitoring performance, maintaining volume health, and following best practices to ensure long-term reliability.
Or read how we handle it in Servers Management.
Related Articles
How to Tune a Linux Server That Falls Over Only at Peak
Load average under one, CPU half idle, and for four minutes every evening the site returns errors. The limits that bite at peak are queues and counters rather than utilisation, which is why no dashboard you own is measuring them. This guide shows where to read the accept queue overflow counter, the PHP-FPM log lines that name the pool ceiling in plain words, the ephemeral port range, and the connection tracking table, along with the sysctl and nginx directives that raise each one. Every observation command comes before the change it justifies.
Server & DevOpsHow to Restore One Table From a Full Cluster Backup
Somebody emptied one table and the rest of the database is still taking orders. Restoring the whole backup over production would throw away every write since the dump, to fix damage that lives in a single table. This is the side restore instead, with the format choice that decides whether selective restore is even possible, the pg_restore flags that behave differently from the pg_dump ones you know, and the foreign keys and sequences that turn a successful restore into a broken application an hour later.
Server & DevOpsNginx vs Caddy in 2026: Which Reverse Proxy Should You Use?
A head-to-head comparison of Nginx and Caddy as reverse proxies in 2026, covering automatic SSL, configuration syntax, performance benchmarks, plugin ecosystems, and guidance on when to choose each.