As North American University IT department we setup a cluster for the need of scalability. Normally with only 1000 students we didn’t need a cluster at all but by the time we decided to give it a try and expose the cluster infrastructure for institutions who are using Moodle.

The Diagram shows  the NAU Network that holds the main servers. Each node is a different Centos 7 server which are in total 11 servers.  Every tool we used on this architecture is open-source.

Servers

The main flow occurs on the NAU side that holds the main servers.

Haproxy3, Web1, Web2, Session

The Public IP is pointed to Haproxy3 which is used to load balance Web1 and Web2 along with roundrobin algorithm. Since Session server is connected to both Web1 and Web2 we used roundrobin instead of source algorithm on Haproxy3 server. This way the same user can have single session regardless load balanced between Web1 or Web2.

NFS, Web1, Web2

While Web1 and Web2 holds the Moodle Web files, they also point to the NFS Server.  NFS keeps the moodle data files on  /home/msen/moodledata along with MUC. Web1 and Web2 are mounted to see the same moodle data folder on NFS. Web1 and Web2 are mounted to the folder of /var/moodledata. 

Haproxy1, Haproxy2, SMTP Relay Server

The flow goes next to the Haproxy1 that holds the VIP (Virtual IP) .  keepalived helps to switch to Haproxy2 in case of Haproxy1 fails but keeps the same VIP.  So when Haproxy1 fails Web1 and Web2 point automatically to the same VIP that is connected to Haproxy2 and vice versa. Haproxy1 and Haproxy2 are two same load balancer servers for database which Haproxy1 is pointed as the start node as load balancer. If one of the haproxy1 or haproxy2 fails, the smtp relay server is triggered and sends a failover email to the defined emails.

MariaDB1, MariaDB2, MariaDB3

Let’s say there is no failure on Haproxy1, then Haproxy1 goes first MariaDB Node1. For extra overload requests, the load balancer Haproxy1 can go to the MariaDB Node 2 and for more extra load Haproxy1 goes to MariaDB Node3.  So the workload is separated through 3 database servers by the roundrobin algorithm on Haproxy1 even for a single user with multiple connections.

MariaDB Node 1, Node 2 and Node 3 rely on the same Galera Wsrep Replicaton Cluster which all of them are masters. Any update, insert or delete, all of the changes reflected to each other. Thanks to Galera Cluster! It replicates every record update on the same level.

Single Sign-On  Office 365

Actually, the users use Office 365 as single sign-on (openID) to all applications including moodle.  The students never to manual sign in for moodle.

---

Run this on each Centos 7 Server

Each server is on Centos 7 and they need to be on the right timezone. So please follow this tutorial to setup the NTP correctly:  CONFIGURE NTP ON CENTOS 7

---

Servers

We have in total 11 servers, each of them runs on CentOS 7.

1. MariaDB Node1
2. MariaDB Node2
3. MariaDB Node3
4. Haproxy1
5. Haproxy2
6. SMTP Relay Server
7. Web1
8. Web2
9. NFS
10. Session
11. Haproxy3

---

Server Specs

	MOODLE CLUSTER	CPU	RAM(Gig)	HD(Gig)
1	db1_10.10.4.1	2	4	30
2	db2_10.10.4.2	2	4	30
3	db3_10.10.4.3	2	4	30
4	haproxy1_10.10.4.15	2	2	10
5	haproxy2_10.10.4.16	2	2	10
6	Virtual IP(VIP) – Reserved- 10.10.4.17 (NOT A SERVER)	–	–	–
7	haproxy3_10.10.4.18	2	2	10
8	nfs_10.10.4.30	1	2	250
9	session_10.10.4.29	2	2	10
10	web1_10.10.4.21	2	12	10
11	web2_10.10.4.22	2	12	10
12	mail_10.10.4.20	1	1	5

---

How to Setup the Moodle Cluster

Let’s go over the cluster architecture step by step starting from bottom to top

1- Galera Wsrep Replication (Node1, Node2, Node3)

Follow this tutorial first: MARIADB CLUSTER ON CENTOS 7

2-  Haproxy for Databases along with Keepalived VIP (Haproxy1, Haproxy2)

Follow this tutorial : INSTALL HAPROXY AND KEEPALIVED ON CENTOS 7 FOR MARIADB CLUSTER

3- Configure SMTP Relay Server

Follow this tutorial: CONFIGURE POSTFIX RELAY FOR SOCKETLABS ON CENTOS 7

4- Install Web Servers for Nau Moodle Cluster (MariaDB Client, NGINX, PHP7)

Follow this tutorial: INSTALL WEB SERVERS FOR NAU MOODLE CLUSTER(MARIADB CLIENT, NGINX, PHP7)

5- Memcached Server  (Session)

Follow this tutorial: INSTALL SEPARATE MEMCACHE SERVER (SESSION) FOR MOODLE ON CENTOS 7

6- Install and Configure NFS Server and Client on Centos 7

Follow this tutorial: INSTALL & CONFIGURE NFS SERVER & CLIENT ON CENTOS 7

(We don’t sync anymore) Feel free to look at this tutorial: INSTALL & RUN & SYNC UNISON ON CENTOS 7

7- Haproxy Layer 7 Load Balancer on CentOS 7 (Haproxy3)

Follow this tutorial: INSTALL HAPROXY LAYER 7 LOAD BALANCER ON CENTOS 7 NGINX  WEBSERVERS

8 – SSL on Haproxy for Centos 7 or Ubuntu 10.04 (Haproxy3)

Follow this tutorial:  INSTALL SSL ON HAPROXY CENTOS 7 OR UBUNTU 14.04 FOR MOODLE

9 – When SSL is Expired for Moodle

Follow this tutorial:  WHEN SSL IS EXPIRED FOR MOODLE