redis集群

发表于 2020-09-18 更新于 2020-12-24 分类于 redis

redis集群

Redis Cluster is a builtin Redis feature that supports automatic sharding, replication and high availability which was previously implemented using Sentinels.

However the cluster stops to operate in the event of larger failures (e.g when the majority of master instances are unavailable). Also, if a master and slave fail at the same time, the cluster cannot continue normal operations (though the workaround is to add more nodes or create an asymmetry in the cluster, to auto-change the cluster layout).

The minimal cluster that works as expected requires to contain at least three master nodes.

key config: cluster-enabled yes

redis-cluster-setup.sh

#!/usr/bin/env bash

for port in 7000 7001 7002 7003 7004 7005
do
  mkdir ${port}
  cd ${port}
  cat >redis.conf << EOF
port ${port}
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

dir ./
loglevel notice
logfile ${port}.log

save 900 1
save 300 10
save 60 10000
EOF
  cd ..
done

This will create 6 directories from 7000 to 7005

step1

Run cd configs;bash ../redis-cluster-setup.sh

Step2

Open 6 tabs, run below cmd

$ cd ???
$ redis-server ./redis.conf

Step3

redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1

The option --cluster-replicas 1 means that we want a slave for every master create

Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002

[OK] All 16384 slots covered.

Step4

check cluster status redis-cli -p 7000 cluster nodes

Step5 testing

$ redis-cli -c -p 7000
redis 127.0.0.1:7000> set foo bar
-> Redirected to slot [12182] located at 127.0.0.1:7002
OK
redis 127.0.0.1:7002> set hello world
-> Redirected to slot [866] located at 127.0.0.1:7000
OK
redis 127.0.0.1:7000> get foo
-> Redirected to slot [12182] located at 127.0.0.1:7002
"bar"
redis 127.0.0.1:7000> get hello
-> Redirected to slot [866] located at 127.0.0.1:7000
"world"

Step6 Fail over

Stop redis on one of master node

redis-cli -p 7001 shutdown

Restart

cd 7001 && redis-server ./redis.conf

Scaling

adding new master

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001

address of the new node as first argument, and the address of a random existing node in the cluster as second argument.

when newnode added, it holds no data as no slots assigned .

resharding

./redis-cli --cluster reshard 127.0.0.1:6001

Prompt: How many slots do you want to move (from 1 to 16384)?

Prompt: what is the target of the resharding, that is, the node that will receive the hash slots.

Promot: et asked from what nodes you want to take those keys. Eg: all means take a bit of hash slots from all the other master nodes.

add new slave

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001 --cluster-slave

./redis-cli --cluster add-node 127.0.0.1:6007 127.0.0.1:6001 --cluster-slave --cluster-master-id 3c3a0c74aae0b56170ccb03a76b60cfe7dc1912e

Remove node

./redis-cli --cluster del-node 127.0.0.1:6001 `<node-id>`

the first argument is just a random node in the cluster, the second argument is the ID of the node you want to remove.

Notes: Remove slave is easy , just use del-node, for master it must be empy before deletion, you need reshard slots to other master before deletion if master is not empy.

https://redis.io/topics/cluster-tutorial#redis-cluster-data-sharding

Setting Up A High Available Multi Node Redis Cluster https://blog.mshimul.com/setting-up-a-high-available-multi-node-redis-cluster/

Creating a Redis Cluster https://medium.com/@iamvishalkhare/create-a-redis-cluster-faa89c5a6bb4

Horizontal scaling in/out techniques for redis cluster https://medium.com/@iamvishalkhare/horizontal-scaling-in-out-techniques-for-redis-cluster-dcd75c696c86

How to Setup a Redis Cluster in CentOS 8 – Part 3 https://www.tecmint.com/setup-redis-cluster-in-centos-8/