二 Redis的两种持久化方式---RDB和AOF

Redis是内存数据库，但是为了防止数据丢失，提供了持久化到硬盘的机制。有RDB和AOF两种持久化方式。

RDB （Redis DataBase）

RDB方式会在指定的时间间隔内，将内存中的数据快照写入磁盘，它恢复时是将数据快照读到内存。

1. Redis RDB配置

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""

save 900 1
save 300 10
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes

# The filename where to dump the DB
dbfilename dump.rdb

# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir /usr/local/var/db/redis/

2. 哪些情况会触发Redis RDB持久化

将配置改成save 60 5，重启redis服务器.

2.1 自动触发---配置文件中的save

也就是说60秒内，如果有5次或以上的写操作，就会触发Redis持久化。配置文件中的save，执行的是bgsave操作，后面会讲到。

#1. 查看rdb快照文件目录，一开始为空，没有dump数据。
$ ls /usr/local/var/db/redis

#2. 在60s内进行5次写操作
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set key1 value1
OK
127.0.0.1:6379> set key1 value2
OK
127.0.0.1:6379> set key1 value3
OK
127.0.0.1:6379> set key1 value4
OK
127.0.0.1:6379> set key1 value5
OK
127.0.0.1:6379> get key1
"value5"

#3. 查看rdb快照目录，生成了dump.rdb文件
$ ls /usr/local/var/db/redis
dump.rdb

#4. 看看dump.rdb文件的内容
$ less /usr/local/var/db/redis/dump.rdb
REDIS0009<FA>   redis-ver^E5.0.3<FA>
redis-bits<C0>@<FA>^Ectime<C2>O<C6>r`<FA>^Hused-mem<C2>`W^O^@<FA>^Laof-preamble<C0>^@<FE>^@<FB>^A^@^@^Dkey1^Fvalue5<FF><A1>
<98>hK<BE>P^A
/usr/local/var/db/redis/dump.rdb (END)

2.2 shutdown命令

正常关闭redis server，也会触发rdb持久化。

#1. 写入一个key
127.0.0.1:6379> set key2 value2
OK

#2. 马上观察dump.rdb，没有更新

#3. shutdown redis server
127.0.0.1:6379> shutdown
not connected>

#4. 再去观察dump.rdb，发现触发了持久化，新写的key被存到了磁盘。
REDIS0009<FA>   redis-ver^E5.0.3<FA>
redis-bits<C0>@<FA>^Ectime<C2>^@<C9>r`<FA>^Hused-mem°^G^P^@<FA>^Laof-preamble<C0>^@<FE>^@<FB>^B^@^@^Dkey2^Fvalue2^@^Dkey1^Fvalue5<FF><98><FE>L1<F7>Sz 
/usr/local/var/db/redis/dump.rdb (END)

注意！！！
意外关闭，例如kill -9，不会触发持久化。

2.3 flushall也会触发持久化，硬盘中的dump.rdb会被清理。

#1. 执行flushall之前，dump文件有119B
redis $ ls -ltrh
total 8
-rw-r--r--  1 a123  admin   119B  4 11 18:14 dump.rdb

#2. 执行flushall操作
127.0.0.1:6379> flushall
OK

#3. 查看dump.rdb，变小成92B，其中数据都被清理掉了。
redis $ ls -ltrh
total 8
-rw-r--r--  1 a123  admin    92B  4 11 18:17 dump.rdb

redis $ less dump.rdb
"dump.rdb" may be a binary file.  See it anyway? 
REDIS0009<FA>   redis-ver^E5.0.3<FA>
redis-bits<C0>@<FA>^Ectime<C2><C3><CC>r`<FA>^Hused-mem<C2>^P^K^P^@<FA>^Laof-preamble<C0>^@<FF>.e1o<E8>  #<ED>
dump.rdb (END)

注意！！！
flushdb不会触发持久化，只是清空内存中的数据，也就是说当前dump.rdb中还是有持久化数据的。不过当后续又触发了rdb持久化时，新的数据快照会被持久化到硬盘，那么flushdb之前的数据就会在硬盘中被清除了。

2.4 执行命令save或者bgsave，也会触发持久化。不过两者略有区别。

save是只管保存，其他不管，全部阻塞(持久化是用的主进程，不会fork子进程). 所以在线上，最好不要使用save命令，太危险。

#1. save命令，触发持久化。
127.0.0.1:6379> save
OK

#2. 观察redis server log
16167:M 11 Apr 2021 18:32:40.092 * DB saved on disk

bgsave redis会在后台异步进行快照操作(fork子进程)，同时可以响应客户端的请求.

#1. bgsave命令
127.0.0.1:6379> bgsave
Background saving started

#2. 查看redis server log，会发现新开启了一个进程来处理持久化操作
16167:M 11 Apr 2021 18:32:54.925 * Background saving started by pid 16264
16264:C 11 Apr 2021 18:32:54.927 * DB saved on disk
16167:M 11 Apr 2021 18:32:54.953 * Background saving terminated with success

3. RDB 自动触发，save m n的实现原理

（下图引用自互联网）

redis将save配置保存到saveparam参数：

save配置保存到saveparam

另外，服务器还保存了dirty计数器和lastsave属性

dirty，lastsave
最后，通过servercron函数默认每隔100毫秒执行一次，检查保存条件是否满足。若满足，则执行持久化，并更新lastsave的时间。

4. RDB的优点与缺点

4.1 优点

rdb备份文件紧凑，全量备份。文件相对不会特别大，恢复起数据来相对较快。
rdb备份时，是fork子进程进行备份操作，不影响父进程，这样redis性能相对较高。

4.2 缺点

如果在下一次持久化之前出现故障，没来得及写入磁盘的数据就会丢失。

AOF （Append Only File）

将我们的所有写命令都记录下来，恢复的时候就把这个文件全部再执行一遍。

1. Redis AOF配置

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# appendfsync always
appendfsync everysec
# appendfsync no

2. 开启AOF，命令被记录到appendonly.aof，同时rdb 持久化也依然生效。

开启AOF的话，默认是appendfsync everysec，也就是每秒都会append到aof文件。
有另一个选项appendfsync always，对于每一次写都会append到aof文件。这种方式会影响redis的性能。

#1. 开启AOF之后，重启redis server，存几个值
redis $ redis-cli -p 6379 -h 127.0.0.1
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set k1 v1
OK
127.0.0.1:6379> set k2 v2
OK
127.0.0.1:6379> set k3 v3
OK
127.0.0.1:6379> set k4 v4
OK
127.0.0.1:6379> set k5 v5

#2. 生成了aof和rdb持久化文件
$ ls
appendonly.aof  dump.rdb

$ vim appendonly.aof
*2
$6
SELECT
$1
0
*3
$3
set
$2
k1
$2
v1
*3
$3
set
$2
k2
$2
v2
*3
$3
set
$2
k3
$2
v3
*3
$3
set
$2
k4
$2
v4
*3
$3
set
$2
k5
$2
v5

3. redis-check-aof

如果aof文件有错误，redis就无法启动。这时可以用redis自带工具 redis-check-aof --fix来修复文件。修复之后就能成功启动redis了。

#1. 在appendonly.aof文件末尾加上一些字符串，来破坏aof文件，内容如下
set
$2
k5
$2
v5
testredischeckaof

#2. 重启redis server,报错
15637:M 15 Apr 2021 22:29:32.481 # Server initialized
15637:M 15 Apr 2021 22:29:32.481 # Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>

#3. 修复aof文件
$ redis-check-aof --fix appendonly.aof
0x              a8: Expected prefix '*', got: 't'
AOF analyzed: size=187, ok_up_to=168, diff=19
This will shrink the AOF from 187 bytes, with 19 bytes, to 168 bytes
Continue? [y/N]: y
Successfully truncated AOF

4. AOF 优缺点

4.1 优点

相对rdb，aof数据安全性更高。aof如果设置为appendfsync everysec，只是有可能会丢一秒的数据。

4.2 缺点

相较于rdb，aof同步文件可能会比较大，因为它记录的是每一个写操作。恢复起来，aof相对也会慢一些。