MySQL group replication plugin won't recognize host's IP address?

Question

I have pulled the current mysql-server image (mysql-server) on 2 Ubuntu 18.04 machines and I want to set up group replication.

This is how I start the mysql container on each machine (--server-id is 2 for the second machine):

docker run -d --name=gr-mysql \
  -v /var/lib/mysql:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=secret -e MYSQL_ROOT_HOST=% \
  -p 3306:3306 -p 33061:33061 \
  mysql/mysql-server:8.0 \
  --port=3306 \
  --log-error-verbosity=3 \
  --server-id=1 \
  --log-bin='mysql-bin-1.log' \
  --enforce-gtid-consistency=ON \
  --log-slave-updates=ON \
  --gtid-mode=ON \
  --transaction-write-set-extraction=XXHASH64 \
  --binlog-checksum=NONE \
  --master-info-repository=TABLE \
  --relay-log-info-repository=TABLE \
  --plugin-load=group_replication.so \
  --relay-log-recovery=ON \
  --loose-group_replication_start_on_boot=OFF \
  --loose-group_replication_group_name=a_valid_uuid \
  --loose-group_replication_local_address=1.2.3.1:33061 \
  --loose-group_replication_group_seeds=1.2.3.1:33061,1.2.3.2:33061 \
  --loose-group_replication_single_primary_mode=OFF \
  --loose-group_replication_enforce_update_everywhere_checks=ON

Now I try to configure group replication on the first machine:

docker exec -t aerobase-mysql mysql -uroot -psecret \
  -e "SET @@GLOBAL.group_replication_bootstrap_group=1;" \
  -e "create user 'repl'@'%';" \
  -e "GRANT REPLICATION SLAVE ON *.* TO repl@'%';" \
  -e "flush privileges;" \
  -e "change master to master_user='repl', master_password='secret' for channel 'group_replication_recovery';" \
  -e "START GROUP_REPLICATION;" \
  -e "SET @@GLOBAL.group_replication_bootstrap_group=0;"

And I'm prompted with this error:

ERROR 3096 (HY000) at line 1: The START GROUP_REPLICATION command failed as there was an error when initializing the group communication layer.

Checking the container log it seems like MySQL can't get the host's IP address:

2020-09-20T22:17:03.060972Z 220 [System] [MY-013587] [Repl] Plugin group_replication reported: 'Plugin 'group_replication' is starting.'
2020-09-20T22:17:03.061056Z 220 [Note] [MY-011716] [Repl] Plugin group_replication reported: 'Current debug options are: 'GCS_DEBUG_NONE'.'
2020-09-20T22:17:03.062114Z 221 [System] [MY-011565] [Repl] Plugin group_replication reported: 'Setting super_read_only=ON.'
2020-09-20T22:17:03.062187Z 220 [Note] [MY-011673] [Repl] Plugin group_replication reported: 'Group communication SSL configuration: group_replication_ssl_mode: "DISABLED"'
2020-09-20T22:17:03.063052Z 220 [Note] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Debug messages will be sent to: asynchronous::/var/lib/mysql/GCS_DEBUG_TRACE'
2020-09-20T22:17:03.063205Z 220 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] There is no local IP address matching the one configured for the local node (1.2.3.1:33061).'
2020-09-20T22:17:03.063415Z 220 [ERROR] [MY-011674] [Repl] Plugin group_replication reported: 'Unable to initialize the group communication engine'
2020-09-20T22:17:03.063434Z 220 [ERROR] [MY-011637] [Repl] Plugin group_replication reported: 'Error on group communication engine initialization'
2020-09-20T22:17:03.063444Z 220 [Note] [MY-011649] [Repl] Plugin group_replication reported: 'Requesting to leave the group despite of not being a member'
2020-09-20T22:17:03.063451Z 220 [ERROR] [MY-011718] [Repl] Plugin group_replication reported: 'Error calling group communication interfaces while trying to leave the group'
2020-09-20T22:17:03.063586Z 221 [System] [MY-011566] [Repl] Plugin group_replication reported: 'Setting super_read_only=OFF.'

In the ifconfig output I can see the host's IP address and that it matches what I entered for MySQL:

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 01:23:45:67:89:00  txqueuelen 0  (Ethernet)
        RX packets 146483  bytes 312211022 (312.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 163698  bytes 12222756 (12.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 1.2.3.1  netmask 255.255.255.0  broadcast 1.2.3.255
        ether 00:11:22:33:44:55  txqueuelen 13888  (Ethernet)
        RX packets 23255307  bytes 10382649061 (10.3 GB)
        RX errors 0  dropped 68  overruns 0  frame 0
        TX packets 393777  bytes 34493066 (34.4 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 144  bytes 7948 (7.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 144  bytes 7948 (7.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

veth0b41841: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 66:55:44:33:22:11  txqueuelen 0  (Ethernet)
        RX packets 944  bytes 2086507 (2.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1058  bytes 81312 (81.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Before I can have group replication I need to make MySQL accept the host's IP. I tried adding an IP whitelist and even using host names instead of IP addresses, to no avail. I don't have firewall or SELinux enabled.

Thanks! (Question has been moved from SO)

How is you _docker_ network configured? Are you sure you can access 1.2.3.0/24 from inside your containers? — mustaccio, Sep 21 '20 at 13:01
Yes, I can ping and connect to each machine both remotely and from each other. Actually the hosts themselves are two virtual machines in an ESX and I use the container for convenience. — towel, Sep 21 '20 at 15:13
Note I've asked about containers, not machines on which they're running. If you can't tell the difference, you probably need to read about Docker networking some more. — mustaccio, Sep 21 '20 at 16:06
Maybe I explained myself incorrectly - there is no docker network. By default the containers have a network bridge to their host and can make any outgoing connection. The containers have no trouble communicating between themselves. For incoming connections to the containers I use static DNS records. The point is that I can't spot any issue from the network side. — towel, Sep 22 '20 at 05:01
Never mind, I think I'll ditch group replication for Galera as I can't get it to work... — towel, Sep 22 '20 at 05:09

score 0 · Answer 1 · answered Jul 26 '21 at 07:12

0

use should notice that your local ip is different with docker's virtual ip. You need to use bash cmd "docker inspect xxx" to find the real virtual ip for mysql image and then config it in local address( replace 1.2.3.1 with docker ip such as 172.23.0.2)

answered Jul 26 '21 at 07:12

wolf289184291

1

This would work only for containers inside the docker network and wouldn't allow for connections to remote machines. I had no problem setting up a single instance using similar configuration (I could see it listening to connections in `netstat`), but couldn't configure group replication. In the end the solution was to switch to MariaDB with Galera. – towel Jul 26 '21 at 08:02
1

you are right, virtual ip is not ok, but you can use docker's host network mode, it will work – wolf289184291 Jul 27 '21 at 09:32

MySQL group replication plugin won't recognize host's IP address?

1 Answers1