mysql主从之半同步复制和lossless无损复制

摘要:
一MySQL的三种复制方式1.1简介asynchronous异步复制fullysynchronous全同步复制Semisynchronous半同步复制从MySQL5.5开始,MySQL以插件的形式支持半同步复制。原理:在异步复制中,master写数据到binlog且sync,slaverequestbinlog后写入relaylog并flushdisk优点:复制的性能最好缺点:master挂掉后,slave可能会丢失数据1.3全同步复制指当主库执行完一个事务,所有的从库都执行了该事务才返回给客户端。

一 MySQL 的三种复制方式

1.1 简介

asynchronous 异步复制

fully synchronous 全同步复制

Semisynchronous 半同步复制

从MySQL5.5 开始,MySQL 以插件的形式支持半同步复制。

mysql主从之半同步复制和lossless无损复制第1张

mysql主从之半同步复制和lossless无损复制第2张

1.2 异步复制(Asynchronous replication)

mysql主从之半同步复制和lossless无损复制第3张

MySQL 默认的复制是异步的,主库在执行完客户端提交的事务后会立即将结果返给给客户端,并不关心从库是否已经接收并处理,这样就会有一个问题,主如果crash 掉了,此时主上已经提交的事务可能并没有传到从上,如果此时,将从提升为主,可能导致新主上的数据不完整。

原理:在异步复制中,master 写数据到binlog 且sync,slaverequest binlog 后写入relay‐log 并flush disk

优点:复制的性能最好

缺点:master 挂掉后,slave 可能会丢失数据

1.3 全同步复制(Fully synchronous replication)

mysql主从之半同步复制和lossless无损复制第4张

指当主库执行完一个事务,所有的从库都执行了该事务才返回给客户端。因为需要等待所有从库执行完该事务才能返回,所以全同步复制的性能必然会收到严重的影响。

优点:数据不会丢失

缺点:会阻塞master session,性能太差,非常依赖网络

1.4 半同步复制(Semisynchronous replication)

mysql主从之半同步复制和lossless无损复制第5张

介于异步复制和全同步复制之间,主库在执行完客户端提交的事务后不是立刻返回给客户端,而是等待至少一个从库接收到并写到relay log 中才返回给客户端。相对于异步复制,半同步复制提高了数据的安全性,同时它也造成了一定程度的延迟,这个延迟最少是一个TCP/IP 往返的时间。所以,半同步复制最好在低延时的网络中使用。

优点:会有数据丢失风险(低)

缺点:会阻塞master session,性能差,非常依赖网络,由于master 是在三段提交的最后commit 阶段完成后才等待,所以master 的其他session 是可以看到这个提交事务的,所以这时候master 上的数据和slave 不一致,master crash 后,slave 数据丢失。

1.5 增强版的半同步复制(lossless replication)

mysql主从之半同步复制和lossless无损复制第6张

原理: 在半同步复制中,master 写数据到binlog 且sync,然后一直等待ACK. 当至少一个slave request bilog 后写入到relay‐log 并flush disk,就返回ack(不需要回放完日志)

优点:数据零丢失(前提是让其一直是lossless replication),

性能好

缺点:会阻塞master session,非常依赖网络由于master 是在三段提交的第二阶段sync binlog 完成后才等待, 所以master 的其他session 是看不见这个提交事务的,所以这时候master 上的数据和slave 一致,master crash 后,slave 没有丢失数据

二 实验

2.1 查看plugins

mysql>show plugins;
+----------------------------+----------+--------------------+---------+---------+
| Name                       | Status   | Type               | Library | License |
+----------------------------+----------+--------------------+---------+---------+
| binlog                     | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| mysql_native_password      | ACTIVE   | AUTHENTICATION     | NULL    | GPL     |
| sha256_password            | ACTIVE   | AUTHENTICATION     | NULL    | GPL     |
| CSV                        | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| MEMORY                     | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| InnoDB                     | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| INNODB_TRX                 | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_LOCKS               | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_LOCK_WAITS          | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP                 | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_RESET           | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMPMEM              | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMPMEM_RESET        | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_PER_INDEX       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_CMP_PER_INDEX_RESET | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_PAGE         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_PAGE_LRU     | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_BUFFER_POOL_STATS   | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_TEMP_TABLE_INFO     | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_METRICS             | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_DEFAULT_STOPWORD | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_DELETED          | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_BEING_DELETED    | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_CONFIG           | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_INDEX_CACHE      | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_FT_INDEX_TABLE      | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_TABLES          | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_TABLESTATS      | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_INDEXES         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_COLUMNS         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_FIELDS          | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_FOREIGN         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_FOREIGN_COLS    | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_TABLESPACES     | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_DATAFILES       | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| INNODB_SYS_VIRTUAL         | ACTIVE   | INFORMATION SCHEMA | NULL    | GPL     |
| MyISAM                     | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| MRG_MYISAM                 | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| PERFORMANCE_SCHEMA         | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| ARCHIVE                    | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| BLACKHOLE                  | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| FEDERATED                  | DISABLED | STORAGE ENGINE     | NULL    | GPL     |
| partition                  | ACTIVE   | STORAGE ENGINE     | NULL    | GPL     |
| ngram                      | ACTIVE   | FTPARSER           | NULL    | GPL     |
+----------------------------+----------+--------------------+---------+---------+

或者查询INFORMATION_SCHEMA.PLUGINS 表

[root@master1 ~]# cd /usr/lib64/mysql/plugin/

[root@master1 plugin]# ll

-rwxr-xr-x. 1 root root   103728 Apr 13 10:36adt_null.so
-rwxr-xr-x. 1 root root   356976 Apr 13 10:36authentication_ldap_sasl_client.so
-rwxr-xr-x. 1 root root    43552 Apr 13 10:36auth_socket.so
-rwxr-xr-x. 1 root root   940312 Apr 13 10:36connection_control.so
drwxr-xr-x. 2 root root     4096 Jul  3 11:31debug
-rwxr-xr-x. 1 root root 21640312 Apr 13 10:36group_replication.so
-rwxr-xr-x. 1 root root   483520 Apr 13 10:36ha_example.so
-rwxr-xr-x. 1 root root   968432 Apr 13 10:36innodb_engine.so
-rwxr-xr-x. 1 root root   957088 Apr 13 10:36keyring_file.so
-rwxr-xr-x. 1 root root   460064 Apr 13 10:36keyring_udf.so
-rwxr-xr-x. 1 root root  1184680 Apr 13 10:36libmemcached.so
-rwxr-xr-x. 1 root root  8973984 Apr 13 10:36libpluginmecab.so
-rwxr-xr-x. 1 root root    21424 Apr 13 10:36locking_service.so
-rwxr-xr-x. 1 root root    53928 Apr 13 10:36mypluglib.so
-rwxr-xr-x. 1 root root    41088 Apr 13 10:36mysql_no_login.so
-rwxr-xr-x. 1 root root 22243648 Apr 13 10:37mysqlx.so
-rwxr-xr-x. 1 root root    49504 Apr 13 10:36rewrite_example.so
-rwxr-xr-x. 1 root root   590936 Apr 13 10:36rewriter.so
-rwxr-xr-x. 1 root root   933904 Apr 13 10:36semisync_master.so    #主库安装
-rwxr-xr-x. 1 root root   159928 Apr 13 10:36semisync_slave.so     #备库安装
-rwxr-xr-x. 1 root root   209520 Apr 13 10:36validate_password.so
-rwxr-xr-x. 1 root root   506320 Apr 13 10:36 version_token.so

2.2 主库配置

查看是否支持动态加载的MySQL 服务器

mysql> show variables like '%dynamic%';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| have_dynamic_loading | YES   |
+----------------------+-------+
1 row in set (0.00sec)
mysql> install plugin rpl_semi_sync_master  soname 'semisync_master.so';   #安装库
Query OK, 0 rows affected (0.00sec)
mysql> show variables like '%rpl_semi%';
+-------------------------------------------+------------+
| Variable_name                             | Value      |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled              | OFF        |     #修改为on状态
| rpl_semi_sync_master_timeout              | 10000      |     #修改为1s
| rpl_semi_sync_master_trace_level          | 32         |
| rpl_semi_sync_master_wait_for_slave_count | 1          |
| rpl_semi_sync_master_wait_no_slave        | ON         |
| rpl_semi_sync_master_wait_point           | AFTER_SYNC |
+-------------------------------------------+------------+

修改my.cnf

[root@master1 ~]# vim /etc/my.cnf

rpl_semi_sync_master_enabled = 1
rpl_semi_sync_master_timeout = 1000

[root@master1 ~]# systemctl restart mysqld

mysql> show variables like '%rpl_semi%';
+-------------------------------------------+------------+
| Variable_name                             | Value      |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled              | ON         |
| rpl_semi_sync_master_timeout              | 1000       |
| rpl_semi_sync_master_trace_level          | 32         |
| rpl_semi_sync_master_wait_for_slave_count | 1          |
| rpl_semi_sync_master_wait_no_slave        | ON         |
| rpl_semi_sync_master_wait_point           | AFTER_SYNC |
+-------------------------------------------+------------+
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name                              | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients               | 0     |
| Rpl_semi_sync_master_net_avg_wait_time     | 0     |
| Rpl_semi_sync_master_net_wait_time         | 0     |
| Rpl_semi_sync_master_net_waits             | 0     |
| Rpl_semi_sync_master_no_times              | 0     |
| Rpl_semi_sync_master_no_tx                 | 0     |
| Rpl_semi_sync_master_status                | ON    |
| Rpl_semi_sync_master_timefunc_failures     | 0     |
| Rpl_semi_sync_master_tx_avg_wait_time      | 0     |
| Rpl_semi_sync_master_tx_wait_time          | 0     |
| Rpl_semi_sync_master_tx_waits              | 0     |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0     |
| Rpl_semi_sync_master_wait_sessions         | 0     |
| Rpl_semi_sync_master_yes_tx                | 0     |
+--------------------------------------------+-------+

rpl_semi_sync_master_timeout

一个以毫秒为单位的值,用于控制主服务器等待来自从服务器的确认提交并恢复到异步复制的时间,超过这个值就是超时。 默认值是10000(10 秒)。超时之后,就从半同步复制,返回到异步复制。

Rpl_semi_sync_master_yes_tx:从库成功确认的提交数量。

Rpl_semi_sync_master_no_tx:从库未成功确认的提交数量。

2.3 备份服务器配置

mysql> show variables like '%dynamic%';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| have_dynamic_loading | YES   |
+----------------------+-------+
mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
mysql> show variables like '%rpl_semi%';
+---------------------------------+-------+
| Variable_name                   | Value |
+---------------------------------+-------+
| rpl_semi_sync_slave_enabled     | OFF   |     #打开为on
| rpl_semi_sync_slave_trace_level | 32    |
+---------------------------------+-------+

修改my.cnf

[root@slave ~]# vim /etc/my.cnf

rpl_semi_sync_slave_enabled = 1

[root@slave ~]# systemctl restart mysqld

mysql> show variables like '%rpl_semi%';
+---------------------------------+-------+
| Variable_name                   | Value |
+---------------------------------+-------+
| rpl_semi_sync_slave_enabled     | ON    |
| rpl_semi_sync_slave_trace_level | 32    |
+---------------------------------+-------+
mysql> show status like '%semi%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| Rpl_semi_sync_slave_status | ON    |
+----------------------------+-------+

2.4 主库验证

mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name                              | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients               | 1     |
| Rpl_semi_sync_master_net_avg_wait_time     | 0     |
| Rpl_semi_sync_master_net_wait_time         | 0     |
| Rpl_semi_sync_master_net_waits             | 2     |
| Rpl_semi_sync_master_no_times              | 1     |
| Rpl_semi_sync_master_no_tx                 | 1     |
| Rpl_semi_sync_master_status                | ON    |
| Rpl_semi_sync_master_timefunc_failures     | 0     |
| Rpl_semi_sync_master_tx_avg_wait_time      | 0     |
| Rpl_semi_sync_master_tx_wait_time          | 0     |
| Rpl_semi_sync_master_tx_waits              | 0     |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0     |
| Rpl_semi_sync_master_wait_sessions         | 0     |
| Rpl_semi_sync_master_yes_tx                | 0     |
+--------------------------------------------+-------+
mysql> insert   into  test values (2);
mysql> insert   into  test values (2);
mysql> insert   into  test values (2);
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name                              | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients               | 1     |
| Rpl_semi_sync_master_net_avg_wait_time     | 0     |
| Rpl_semi_sync_master_net_wait_time         | 0     |
| Rpl_semi_sync_master_net_waits             | 5     |
| Rpl_semi_sync_master_no_times              | 1     |
| Rpl_semi_sync_master_no_tx                 | 1     |
| Rpl_semi_sync_master_status                | ON    |
| Rpl_semi_sync_master_timefunc_failures     | 0     |
| Rpl_semi_sync_master_tx_avg_wait_time      | 1013  |
| Rpl_semi_sync_master_tx_wait_time          | 3041  |
| Rpl_semi_sync_master_tx_waits              | 3     |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0     |
| Rpl_semi_sync_master_wait_sessions         | 0     |
| Rpl_semi_sync_master_yes_tx                | 3     |    #增加数据.这个之会增加
+--------------------------------------------+-------+

从端数据已经同步

mysql> select *from master1.test;
+------+
| id   |
+------+
|    1 |
|    2 |
|    2 |
|    2 |
+------+

2.5测试AFTER_SYNC 和AFTER_COMMIT

主库设置超时时间为1000 秒,备库停掉复制,模拟timeout

mysql> set global rpl_semi_sync_master_timeout=1000000;
mysql>stop slave;
mysql> insert   into  test values (11);   #会一直卡住
mysql> select *from master1.test;
+------+
| id   |
+------+
|    1 |
|    2 |
|    2 |
|    2 |
+------+

重启主库数据库,模拟主库宕机,从看数据记录

[root@master1 ~]# systemctl start mysqld
[root@master1 ~]# mysql -uroot -p123456
mysql> select *from master1.test;
+------+
| id   |
+------+
|    1 |
|    2 |
|    2 |
|    2 |
|   11 |   #主库有记录
+------+

备库开启slave

mysql>start slave;
Query OK, 0 rows affected (0.00sec)
mysql> select *from master1.test;
+------+
| id   |
+------+
|    1 |
|    2 |
|    2 |
|    2 |
|   11 |     #数据已经同步,没有丢失
+------+

无损的半同步复制是在write binlog 之后。需要得到备库的确认。所以这时候主库宕机,不会发生丢数据。当主库启动后,插入的数据重新可见。

将rpl_semi_sync_master_wait_point 设置为AFTER_COMMIT,

再次测试:

主库设置超时时间为1000 秒,备库停掉复制,模拟timeout

mysql> set global rpl_semi_sync_master_wait_point=AFTER_COMMIT;
mysql> set global rpl_semi_sync_master_timeout=1000000;
mysql> show variables like '%semi%';
+-------------------------------------------+--------------+
| Variable_name                             | Value        |
+-------------------------------------------+--------------+
| rpl_semi_sync_master_enabled              | ON           |
| rpl_semi_sync_master_timeout              | 1000000      |
| rpl_semi_sync_master_trace_level          | 32           |
| rpl_semi_sync_master_wait_for_slave_count | 1            |
| rpl_semi_sync_master_wait_no_slave        | ON           |
| rpl_semi_sync_master_wait_point           | AFTER_COMMIT |
+-------------------------------------------+--------------+
mysql> insert into master1.test values (12); #一直卡住
[root@master1 ~]# mysql -uroot -p123456 #另开一个窗口,发现已经有12这个数据 mysql> select *from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | | 11 | | 12 | +------+

这样当从库起来之后,数据已经提交,从库就会缺少这个数据

再开一个窗口查询这条数据,发现可以查询到。这时候主库宕机,会发生数据丢失。

主库重新启动,备库启动slave 会同步到备库。

免责声明:文章转载自《mysql主从之半同步复制和lossless无损复制》仅用于学习参考。如对内容有疑问,请及时联系本站处理。

上篇国密证书生成实践多种方式实现千位分隔符下篇

宿迁高防,2C2G15M,22元/月;香港BGP,2C5G5M,25元/月 雨云优惠码:MjYwNzM=

相关文章

macOS + MySql8 问题

最近mac 升级安装mysql后遇到一系列问题,稍作记录。 1、升级8以后,使用sequelpro 连接报错 MySQL said: Authentication plugin 'caching_sha2_password' cannot be loaded: dlopen(/usr/local/mysql/lib/plugin/caching_sha2_...

MyBatis(三)全局配置文件 之 databaseProvider 数据库厂商标识

databaseIdProvider环境一、databaseIdProvider 属性 MyBatis可以根据不同的数据库厂商执行不同的语句 <databaseIdProvider type="DB_VENDOR"> <!-- 为不同的数据库厂商起别名 --> <property name="M...

Windows系统下MySQL添加到系统服务方法(mysql解压版)

MySQL软件版本:64位 5.7.12 1、首先配置MySQL的环境变量,在系统环境变量Path的开头添加MySQL的bin目录的路径,以“;”结束,我的路径配置如下: 2、修改MySQL根目录下的my-default.ini文件的内容;主要修改2个属性:basedir和datadir。 basedir:MySQL的根目录; datadir:数据库数据...

MySQL开启general_log并设置路径

在本地开发的时候,有时需要临时开启MySQL的全局log,查看每条sql执行记录。 以下操作直接在mysql命令行里面执行,不需重启mysql。需要用root身份执行命令。 mysql> set global general_log=on;Query OK, 0 rows affected (0.02 sec)mysql> use l5mDat...

mysql 字符串拼接

CONCAT() CONCAT_WS() GROUP_CONCAT() 一、CONCAT() : 最常用的字符串拼接方法,但遇到拼接中的字符串出现null的情况会返回null 语法:CONCAT(string1,string2) DEMO1 mysql > SELECT CONCAT(s_name,s_sex) FROM student +-...

Python3 pymysql 访问MySql数据库

使用pymysql来访问MySQL数据库,注意,有些使用PyMySQL,Python2是mysqldb 如没该module则安装:pip install pymysql (本人Java转Python,还有java编写习惯,有错误请及时提出,相互交流) 1、定义获取数据库连接的Module,  DataBase.py import pymysql clas...