1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181
|
#
# This include file is used by more than one test suite
# (currently rpl and binlog_encryption).
# Please check all dependent tests after modifying it
#
############################################################
# Purpose: WL#5064 Testing with corrupted events.
# The test emulates the corruption at the vary stages
# of replication:
# - in binlog file
# - in network
# - in relay log
############################################################
#
# The tests intensively utilize @@global.debug_dbug. Note,
# Bug#11765758 - 58754,
# @@global.debug_dbug is read by the slave threads through dbug-interface.
# Hence, before a client thread set @@global.debug_dbug we have to ensure that:
# (a) the slave threads are stopped, or (b) the slave threads are in
# sync and waiting.
--source include/have_debug.inc
--source include/master-slave.inc
--connection slave
--source include/stop_slave.inc
CHANGE MASTER TO MASTER_USE_GTID=NO;
--source include/start_slave.inc
--connection master
# Block legal errors for MTR
call mtr.add_suppression('Found invalid event in binary log');
call mtr.add_suppression('Slave I/O: Relay log write failure: could not queue event from master');
call mtr.add_suppression('event read from binlog did not pass crc check');
call mtr.add_suppression('Replication event checksum verification failed');
call mtr.add_suppression('Event crc check failed! Most likely there is event corruption');
call mtr.add_suppression('Slave SQL: Error initializing relay log position: I/O error reading event at position .*, error.* 1593');
SET @old_master_verify_checksum = @@master_verify_checksum;
# Creating test table/data and set corruption position for testing
--echo # 1. Creating test table/data and set corruption position for testing
--connection master
--echo * insert/update/delete rows in table t1 *
# Corruption algorithm modifies only the first event and
# then will be reset. To avoid checking always the first event
# from binlog (usually it is FD) we randomly execute different
# statements and set position for corruption inside events.
CREATE TABLE t1 (a INT NOT NULL PRIMARY KEY, b VARCHAR(10), c VARCHAR(100));
--disable_query_log
let $i=`SELECT 3+CEILING(10*RAND())`;
let $j=1;
let $pos=0;
while ($i) {
eval INSERT INTO t1 VALUES ($j, 'a', NULL);
if (`SELECT RAND() > 0.7`)
{
eval UPDATE t1 SET c = REPEAT('a', 20) WHERE a = $j;
}
if (`SELECT RAND() > 0.8`)
{
eval DELETE FROM t1 WHERE a = $j;
}
if (!$pos) {
let $pos= query_get_value(SHOW MASTER STATUS, Position, 1);
--sync_slave_with_master
--source include/stop_slave.inc
--disable_query_log
--connection master
}
dec $i;
inc $j;
}
--enable_query_log
# Emulate corruption in binlog file when SHOW BINLOG EVENTS is executing
--echo # 2. Corruption in master binlog and SHOW BINLOG EVENTS
SET @saved_dbug = @@global.debug_dbug;
SET @@global.debug_dbug="d,corrupt_read_log_event_char";
--echo SHOW BINLOG EVENTS;
--disable_query_log
send_eval SHOW BINLOG EVENTS FROM $pos;
--enable_query_log
--error ER_ERROR_WHEN_EXECUTING_COMMAND
reap;
SET @@global.debug_dbug=@saved_dbug;
# Emulate corruption on master with crc checking on master
--echo # 3. Master read a corrupted event from binlog and send the error to slave
# We have a rare but nasty potential race here: if the dump thread on
# the master for the _old_ slave connection has not yet discovered
# that the slave has disconnected, we will inject the corrupt event on
# the wrong connection, and the test will fail
# (+d,corrupt_read_log_event2 corrupts only one event).
# So kill any lingering dump thread (we need to kill; otherwise dump thread
# could manage to send all events down the socket before seeing it close, and
# hang forever waiting for new binlog events to be created).
let $id= `select id from information_schema.processlist where command = "Binlog Dump"`;
if ($id)
{
--disable_query_log
--error 0,1094
eval kill $id;
--enable_query_log
}
let $wait_condition=
SELECT COUNT(*)=0 FROM INFORMATION_SCHEMA.PROCESSLIST WHERE command = 'Binlog Dump';
--source include/wait_condition.inc
SET @@global.debug_dbug="d,corrupt_read_log_event2_set";
--connection slave
START SLAVE IO_THREAD;
let $slave_io_errno= 1236;
--let $slave_timeout= 10
--source include/wait_for_slave_io_error.inc
--connection master
SET @@global.debug_dbug=@saved_dbug;
# Emulate corruption on master without crc checking on master
--echo # 4. Master read a corrupted event from binlog and send it to slave
--connection master
SET GLOBAL master_verify_checksum=0;
SET @@global.debug_dbug="d,corrupt_read_log_event2_set";
--connection slave
START SLAVE IO_THREAD;
# When the checksum error is detected, the slave sets error code 1743
# (ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE) in queue_event(), then immediately
# sets error 1595 (ER_SLAVE_RELAY_LOG_WRITE_FAILURE) in handle_slave_io().
# So we usually get 1595, but it is occasionally possible to get 1743.
let $slave_io_errno= 1595,1743; # ER_SLAVE_RELAY_LOG_WRITE_FAILURE, ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE
--source include/wait_for_slave_io_error.inc
--connection master
SET @@global.debug_dbug=@saved_dbug;
SET GLOBAL master_verify_checksum=1;
# Emulate corruption in network
--echo # 5. Slave. Corruption in network
--connection slave
SET @saved_dbug_slave = @@GLOBAL.debug_dbug;
SET @@global.debug_dbug="d,corrupt_queue_event";
START SLAVE IO_THREAD;
let $slave_io_errno= 1595,1743; # ER_SLAVE_RELAY_LOG_WRITE_FAILURE, ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE
--source include/wait_for_slave_io_error.inc
SET @@global.debug_dbug=@saved_dbug_slave;
# Emulate corruption in relay log
--echo # 6. Slave. Corruption in relay log
SET @@global.debug_dbug="d,corrupt_read_log_event_char";
START SLAVE SQL_THREAD;
let $slave_sql_errno= 1593;
--source include/wait_for_slave_sql_error.inc
SET @@global.debug_dbug=@saved_dbug_slave;
# Start normal replication and compare same table on master
# and slave
--echo # 7. Seek diff for tables on master and slave
--connection slave
--source include/start_slave.inc
--connection master
--sync_slave_with_master
let $diff_tables= master:test.t1, slave:test.t1;
--source include/diff_tables.inc
# Clean up
--echo # 8. Clean up
--connection master
set @@global.debug_dbug = @saved_dbug;
SET GLOBAL master_verify_checksum = @old_master_verify_checksum;
DROP TABLE t1;
--sync_slave_with_master
--source include/rpl_end.inc
|