HAProxy - Application WebUI Server Load Balancer

HAProxy (WebUI Load Balancing) Server Setup

The purpose of this page is to help you install/setup haproxy on a dedicated machine to load balance two or more application servers. This section/page only discusses the haproxy config. iDempiere also needs to be configured so that services only run on one of the application servers. HAProxy is an open source load balancer that I have used in the past with good success. AWS also has a load balance service. Here is an example where the services server sits inside the load balanced cluster. . 10.6.1.230 HAProxy . / | \ 10.6.1.231 10.6.1.232 10.6.1.233(main iD/AD sever) Here is an example where the services server sits outside the load balanced cluster (recommended). If any load balanced server goes down, users of that server will lose their session. They will simply be asked to re-login. All other users will be ignorant to the situation. User load should not affect services. . 10.6.1.230 HAProxy . / \ 10.6.1.231 10.6.1.232 10.6.1.233(main iD/AD sever)

haproxy Installation Steps

Create a new Ubuntu 20.04 server. Here is the most updated copy of the HA Proxy config. The following commands install and configure haproxy.

sudo apt-get update

sudo apt-get install haproxy socat

sudo mv /etc/haproxy/haproxy.cfg{,.original}

Open a new haproxy config file and copy this HA Proxy config contents into the file. Look for the below bolded lines and update according to your situation.

Sample - use above link for actual config

global
    log 127.0.0.1 local0 notice
    maxconn 2000
    user haproxy
    group haproxy
    stats socket /etc/haproxy/haproxysock level admin

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    option redispatch
    timeout connect  2m
    timeout client  3m
    timeout server  3m

listen idempiere
    bind 0.0.0.0:80
    mode http
    stats enable
    stats uri /haproxy?stats
    stats realm Strictly\ Private
    stats auth admin:ChangeMeN0w111
    stats admin if TRUE
    balance leastconn
    option httpchk
    option http-server-close
    option forwardfor
    stick-table type ip size 1m expire 12h 
    stick on src
    server idempiere1 172.30.0.105:8080 check
    server idempiere2 172.30.0.99:8080 check

Stick-table discussion Balancing on IP discussion Start haproxy.

sudo service haproxy restart

Historical Reference - installation for 16.04

SSL with HAProxy

Good reference includes ssl termination and forcing http to https

External Health Check

Write an external health check

Troubleshooting HAProxy

Monitoring

http://IP_OF_SERVER/haproxy?stats

Configuration Issues

If you have issues starting HAProxy, use the following command to start it and see the error:

haproxy -f /etc/haproxy/haproxy.cfg -c

Reviewing logs

Count requests per IP:

grep -E '(19:[0-5][0-9]:[0-5][0-9]|20:20:00)' /var/log/haproxy.log | awk '{print $6}' | cut - d ':' -f1 | sort | uniq -c | sort

first column - number of requests second column - IP from which requests came

Cookie or Session Issues

If you are having trouble getting iDempiere to behave correctly behind HAProxy, make sure your "stick on req.cook(<cookie_name>)" value above is correct. An easy way to know that your session ID is incorrect is that your login => Username and Password fields will be grayed out. iDempiere changes the cookie name from version to version. It has gone from JSESSIONID to WEBUI_SESSIONID to ROOT_SESSIONID. To know for sure, use your browser developer tools => storage options to see the cookies that are saved by iDempiere.

Control HAProxy from Command Line (CLI)

echo "disable server yourbackendname/yourservername" | socat stdio /etc/haproxy/haproxysock

To Enable, replace "disable" with "enable" in the command above. More details...

HAProxy User Interface

This project looks promising; however, I have not used it yet.

Note About Load Balancing

If you are using HAProxy for load balancing, you need to review your DB Server => /etc/postgresql/x.x/main/postgresql.conf => max_connections setting compared to the application server => /opt/idempiere-server/PostgreSQL/pool.properties. Four servers each running a pool.properties => 90 max connections is a lot of connections for the DB.

Session Details

To learn more about the appsession (cookie), see Chrome => Developer Tools (right-click inspect option) => Application menu => Storage section => Cookies page. You can change the server details located to the left of the login screen by finding a file containing the text ">Database Version<" and modifying it to represent something unique to that server ">Database Version A<" and ">Database Version B<". Here is the command to find the file you want to change. Please keep in mind that we are not creating two database versions. In fact, both servers should be touching the same database. I was just looking for an easy way to distinguish the two application servers. Note: this search might take a while to complete. In my server, it is located in: /opt/idempiere-server/plugins/org.adempiere.ui.zk_3.1.0.v20180301-1816/theme/default/zul/login/version-info.zul.

grep -rnw /opt/idempiere-server/ -e "Database Version"