Setting up an tunne GlusterFS on Ubuntu 22.04
GlusterFS is a scalable, distributed filesystem that ensures high availability and fault tolerance. This guide walks you through configuring GlusterFS on three Ubuntu 22.04 servers, enabling a replicated filesystem across nodes. You'll also learn maintenance tasks, performance optimizations, and troubleshooting techniques for production-ready setups.
Overview
We will configure GlusterFS across three servers:
- web: 10.0.0.2
- web-node1: 10.0.0.5
- web-node2: 10.0.0.6
By replicating data across these nodes, GlusterFS ensures fault tolerance and consistent access to your data, even in case of node failures.
Prerequisites
- Three Ubuntu 22.04 servers with sudo privileges.
- Network connectivity between the servers.
- Update the
/etc/hostsfile on each server with the following entries:
10.0.0.5 web-node1
10.0.0.6 web-node2
Installation Steps
Step 1: Install GlusterFS on All Nodes
sudo apt-get install glusterfs-server -y
sudo systemctl start glusterd
sudo systemctl enable glusterd
Step 2: Configure the GlusterFS Trusted Pool
On the primary node (web), add the other nodes to the trusted pool:
sudo gluster peer probe web-node2
sudo gluster peer status
gluster peer status on any node to verify that all peers are connected.
Step 3: Create Brick Directories
Create brick directories on all nodes to store GlusterFS data:
Step 4: Create and Start the Volume
On the primary node, create a replicated GlusterFS volume:
sudo gluster volume start gv0
Step 5: Mount the GlusterFS Volume
Install the GlusterFS client and mount the volume on each node:
sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs web:/gv0 /mnt/glusterfs
To auto-mount at boot, add the following line to /etc/fstab:
Testing the Setup
Create a test file on one node to verify data replication:
Check for the file's presence on all nodes:
Adding a New Node
Step 1: Prepare the New Node
Install GlusterFS and add its IP to /etc/hosts on all servers:
Step 2: Add the Node to the Trusted Pool
sudo gluster peer status
Network Optimization
Enhance performance by tuning TCP settings:
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_rmem='4096 87380 16777216'
sudo sysctl -w net.ipv4.tcp_wmem='4096 65536 16777216'
sudo sysctl -w net.core.netdev_max_backlog=250000
GlusterFS Maintenance
Rebalancing Volumes
Rebalance the volume after adding or removing bricks:
Checking Volume Health
Monitor volume health and heal inconsistencies:
sudo gluster volume heal gv0
Troubleshooting
Peer Not Connected
Verify:
- Network connectivity between nodes.
- Ports 24007-24009 are open.
- The
glusterdservice is running on all nodes.
So ..
By following this guide, you've set up a distributed and replicated GlusterFS filesystem across multiple nodes. This configuration ensures high availability and fault tolerance, making it suitable for demanding environments. Regularly monitor performance, maintain volume health, and follow best practices to ensure long-term reliability.
