Setting up GlusterFS on Ubuntu 22.04
Overview
This guide covers setting up GlusterFS, a scalable network filesystem, on three Ubuntu 22.04 servers (web, web-node1, and web-node2). GlusterFS allows you to create a distributed filesystem that replicates data across multiple nodes for high availability and fault tolerance.
Prerequisites
Three Ubuntu 22.04 servers:
- web (10.0.0.2)
- web-node1 (10.0.0.5)
- web-node2 (10.0.0.6)
Add these entries to the end of the /etc/hosts file on each server:
10.0.0.5 web-node1
10.0.0.6 web-node2
Installation Steps
1. Install GlusterFS on All Nodes
sudo apt-get install glusterfs-server -y
sudo systemctl start glusterd
sudo systemctl enable glusterd
2. Configure the GlusterFS Trusted Pool
On web, add the other nodes:
sudo gluster peer probe web-node2
sudo gluster peer status
3. Create Brick Directories
On all nodes, create the directory for GlusterFS bricks:
4. Create and Start the Volume
On web, create the volume with replica configuration:
sudo gluster volume start gv0
5. Mount the GlusterFS Volume
On each node, install the GlusterFS client and mount the volume:
sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs web:/gv0 /mnt/glusterfs
To mount the volume automatically at boot, add the following line to /etc/fstab:
Testing the Setup
Create a Test File: On one of the nodes, create a file on the mounted volume:
Verify Replication: Check if the file is present on all nodes:
Explanation
- GlusterFS Installation: Installs the GlusterFS server software to manage the distributed filesystem.
- Trusted Pool Configuration: Sets up a trusted relationship between nodes to allow data replication.
- Brick Directory Creation: Creates directories that serve as bricks (storage units) for GlusterFS.
- Volume Creation and Start: Creates a replicated volume for data redundancy.
- Mounting the Volume: Attaches the volume to a mount point, enabling file operations.
- Testing: Verifies that data written to one node is replicated across all nodes.
Adding a New Node to GlusterFS
To add a new node (web-node3) to the GlusterFS setup, follow these steps:
1. Prepare the New Node
Ensure the new node (web-node3) has Ubuntu 22.04 installed.
2. Update Hosts File
Add the new node’s IP and hostname to /etc/hosts on all existing nodes:
3. Install GlusterFS on web-node3
sudo apt-get install glusterfs-server -y
sudo systemctl start glusterd
sudo systemctl enable glusterd
4. Add web-node3 to the Trusted Pool
sudo gluster peer status
5. Create Brick Directory on web-node3
6. Add the New Brick to the Volume
7. Rebalance the Volume
8. Mount the Volume on web-node3
sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs web:/gv0 /mnt/glusterfs
To auto-mount, add to /etc/fstab on web-node3:
Network Optimization
1. Increase TCP Buffer Size
sudo sysctl -w net.core.wmem_max=16777216
sudo sysctl -w net.ipv4.tcp_rmem=’4096 87380 16777216′
sudo sysctl -w net.ipv4.tcp_wmem=’4096 65536 16777216′
sudo sysctl -w net.core.netdev_max_backlog=250000
GlusterFS Volume Example Options
Enable Performance Translators:
sudo gluster volume set gv0 performance.write-behind-window-size 4MB
sudo gluster volume set gv0 performance.read-ahead on
sudo gluster volume set gv0 performance.io-cache on
sudo gluster volume set gv0 performance.quick-read on
sudo gluster volume set gv0 performance.stat-prefetch on
Tuning I/O Threads
By following these steps, you can expand your GlusterFS cluster to include new nodes, ensuring continued scalability and reliability. This setup offers high availability and fault tolerance, making it ideal for environments requiring robust data replication and access.
