备注: watchdog可以避免 pgpool 的单点故障。
参考文章：http://francs3.blog.163.com/blog/static/4057672720149285445881/
一、准备工作
1、pgpool的安装工作，可以参考我另外一篇pgpoll连接池模式
2、pg9.5的主备配置，可以参考主备的配置
3、ssh-keygen信任关系，参考pg9.5主备中的，基本就是双机root\postgres两个用户互相信任，但是千万别忘了一自己信任自己，例如failer_command他会ssh 新pri主机IP，如果不信任自己。。。登着郁闷吧
4、依赖包apt-get install arping
5、修改两边的/etc/hosts，否则，在failer_command执行的时候会找不到最新主服务的HOSTNAME:
192.168.180.222 master
192.168.180.223 slave
二、数据库recovery.conf 配置（apt-get的目录在/var/lib/postgresql/9.5/main）
restore_command = 'cp /var/lib/postgresql/archive/%f %p'standby_mode = 'on'primary_conninfo= 'host=192.168.180.222 port=5432 user=postgres password=postgres'trigger_file='/tmp/trigger_file0'
备注: primary_conninfo 的 host 分别配置对端 host IP
三、master的pgpool.conf配置
grep ^[a-z] pgpool.conf
listen_addresses = '*' #所有地址都可以访问port = 9999 #端口自定义 socket_dir = '/var/run/postgresql' #pg建议放在/var/run/postgresqllisten_backlog_multiplier = 2serialize_accept = offpcp_listen_addresses = '*'pcp_port = 9898pcp_socket_dir = '/var/run/postgresql' #pg建议放在/var/run/postgresqlbackend_hostname0 = '192.168.180.222' #配置数据节点backend_port0 = 5432backend_weight0 = 1backend_data_directory0 = '/var/lib/postgresql/9.5/main'backend_flag0 = 'ALLOW_TO_FAILOVER'backend_hostname1 = '192.168.180.223' #配置数据节点backend_port1 = 5432backend_weight1 = 1backend_data_directory1 = '/var/lib/postgresql/9.5/main'backend_flag1 = 'ALLOW_TO_FAILOVER'enable_pool_hba = onpool_passwd = 'pool_passwd'authentication_timeout = 60ssl = offnum_init_children = 32max_pool = 4child_life_time = 300child_max_connections = 0connection_life_time = 450client_idle_limit = 300log_destination = 'syslog'log_line_prefix = '%t: pid %p: ' log_connections = offlog_hostname = offlog_statement = offlog_per_node_statement = offlog_standby_delay = 'none'syslog_facility = 'LOCAL0'syslog_ident = 'pgpool'debug_level = 0pid_file_name = '/var/run/pgpool/pgpool.pid'logdir = '/tmp'connection_cache = onreset_query_list = 'ABORT; DISCARD ALL'replication_mode = offreplicate_select = offinsert_lock = offlobj_lock_table = ''replication_stop_on_mismatch = offfailover_if_affected_tuples_mismatch = offload_balance_mode = onignore_leading_white_space = onwhite_function_list = ''black_function_list = 'currval,lastval,nextval,setval'database_redirect_preference_list = ''app_name_redirect_preference_list = ''allow_sql_comments = offmaster_slave_mode = on # 设置流复制模式master_slave_sub_mode = 'stream' # 设置流复制模式，采用postgresql自身的流复制sr_check_period = 5sr_check_user = 'postgres'sr_check_password = 'postgres'sr_check_database = 'postgres'delay_threshold = 0follow_master_command = ''health_check_period = 1 #健康检查开启health_check_timeout = 10 #10秒超时就算挂了health_check_user = 'postgres' #检查的用户health_check_password = 'postgres' #检查的密码health_check_database = 'postgres' #检查的数据库health_check_max_retries = 1 #最大重试次数health_check_retry_delay = 2 #每次重试延迟connect_timeout = 10000 failover_command = '/usr/lib/postgresql/9.5/bin/failover_stream.sh %d %H /tmp/trigger_file0' #配置 failover 脚本，脚本内容下面会贴出。failback_command = ''fail_over_on_backend_error = onsearch_primary_node_timeout = 10recovery_user = 'postgres'recovery_password = 'postgres'recovery_1st_stage_command = ''recovery_2nd_stage_command = ''recovery_timeout = 90client_idle_limit_in_recovery = 0use_watchdog = on #使用watch dogtrusted_servers = ''ping_path = '/bin'wd_hostname = '192.168.180.222' #配置watch dog绑定的IPwd_port = 9000 #配置watch dog端口wd_priority = 1wd_authkey = ''wd_ipc_socket_dir = '/tmp'delegate_IP = '192.168.180.221' #配置 pgpool 的 VIP，避免 pgpool 的单点故障if_cmd_path = '/sbin' #以下几个网卡命令不需要修改if_up_cmd = 'ip addr add $_IP_$/24 dev eth0 label eth0:0'if_down_cmd = 'ip addr del $_IP_$/24 dev eth0'arping_path = '/usr/sbin'arping_cmd = 'arping -U $_IP_$ -w 1'clear_memqcache_on_escalation = onwd_escalation_command = ''wd_de_escalation_command = ''wd_monitoring_interfaces_list = '' wd_lifecheck_method = 'heartbeat'wd_interval = 10wd_heartbeat_port = 9694wd_heartbeat_keepalive = 2wd_heartbeat_deadtime = 30heartbeat_destination0 = '192.168.180.223' #配置对端的 hostnameheartbeat_destination_port0 = 9694 #配置对端的 hostnameheartbeat_device0 = 'eth0' #配置对端的 hostnamewd_life_point = 3wd_lifecheck_query = 'SELECT 1'wd_lifecheck_dbname = 'postgres'wd_lifecheck_user = 'postgres'wd_lifecheck_password = 'postgres'other_pgpool_hostname0 = '192.168.180.223' #配置对端的 pgpoolother_pgpool_port0 = 9999 #配置对端的 pgpoolother_wd_port0 = 9000 #配置对端的 pgpool-wacthdogrelcache_expire = 0relcache_size = 256check_temp_table = oncheck_unlogged_table = onmemory_cache_enabled = offmemqcache_method = 'shmem'memqcache_memcached_host = 'localhost'memqcache_memcached_port = 11211memqcache_total_size = 67108864memqcache_max_num_cache = 1000000memqcache_expire = 0memqcache_auto_cache_invalidation = onmemqcache_maxcache = 409600memqcache_cache_block_size = 1048576memqcache_oiddir = '/var/log/pgpool/oiddir'white_memqcache_table_list = ''black_memqcache_table_list = ''
四、slave pgpool.conf的配置
listen_addresses = '*'port = 9999socket_dir = '/var/run/postgresql'listen_backlog_multiplier = 2serialize_accept = offpcp_listen_addresses = '*'pcp_port = 9898pcp_socket_dir = '/var/run/postgresql'backend_hostname0 = '192.168.180.222'backend_port0 = 5432backend_weight0 = 1backend_data_directory0 = '/var/lib/postgresql/9.5/main'backend_flag0 = 'ALLOW_TO_FAILOVER'backend_hostname1 = '192.168.180.223'backend_port1 = 5432backend_weight1 = 1backend_data_directory1 = '/var/lib/postgresql/9.5/main'backend_flag1 = 'ALLOW_TO_FAILOVER'enable_pool_hba = onpool_passwd = 'pool_passwd'authentication_timeout = 60ssl = offnum_init_children = 32max_pool = 4child_life_time = 300child_max_connections = 0connection_life_time = 450client_idle_limit = 300log_destination = 'syslog'log_line_prefix = '%t: pid %p: ' log_connections = offlog_hostname = offlog_statement = offlog_per_node_statement = offlog_standby_delay = 'none'syslog_facility = 'LOCAL0'syslog_ident = 'pgpool'debug_level = 0pid_file_name = '/var/run/pgpool/pgpool.pid'logdir = '/tmp'connection_cache = onreset_query_list = 'ABORT; DISCARD ALL'replication_mode = offreplicate_select = offinsert_lock = offlobj_lock_table = ''replication_stop_on_mismatch = offfailover_if_affected_tuples_mismatch = offload_balance_mode = onignore_leading_white_space = onwhite_function_list = ''black_function_list = 'currval,lastval,nextval,setval'database_redirect_preference_list = ''app_name_redirect_preference_list = ''allow_sql_comments = offmaster_slave_mode = onmaster_slave_sub_mode = 'stream'sr_check_period = 5sr_check_user = 'postgres'sr_check_password = 'postgres'sr_check_database = 'postgres'delay_threshold = 0follow_master_command = ''health_check_period = 1health_check_timeout = 10health_check_user = 'postgres'health_check_password = 'postgres'health_check_database = 'postgres'health_check_max_retries = 1health_check_retry_delay = 2connect_timeout = 10000failover_command = '/usr/lib/postgresql/9.5/bin/failover_stream.sh %d %H /tmp/trigger_file0'failback_command = ''fail_over_on_backend_error = onsearch_primary_node_timeout = 10recovery_user = 'postgres'recovery_password = 'postgres'recovery_1st_stage_command = ''recovery_2nd_stage_command = ''recovery_timeout = 90client_idle_limit_in_recovery = 0use_watchdog = ontrusted_servers = ''ping_path = '/bin'wd_hostname = '192.168.180.223'wd_port = 9000wd_priority = 1wd_authkey = ''wd_ipc_socket_dir = '/tmp'delegate_IP = '192.168.180.221'if_cmd_path = '/sbin'if_up_cmd = 'ip addr add $_IP_$/24 dev eth0 label eth0:0'if_down_cmd = 'ip addr del $_IP_$/24 dev eth0'arping_path = '/usr/sbin'arping_cmd = 'arping -U $_IP_$ -w 1'clear_memqcache_on_escalation = onwd_escalation_command = ''wd_de_escalation_command = ''wd_monitoring_interfaces_list = '' wd_lifecheck_method = 'heartbeat'wd_interval = 10wd_heartbeat_port = 9694wd_heartbeat_keepalive = 2wd_heartbeat_deadtime = 30heartbeat_destination0 = '192.168.180.222'heartbeat_destination_port0 = 9694 heartbeat_device0 = 'eth0'wd_life_point = 3wd_lifecheck_query = 'SELECT 1'wd_lifecheck_dbname = 'postgres'wd_lifecheck_user = 'postgres'wd_lifecheck_password = 'postgres'other_pgpool_hostname0 = '192.168.180.222'other_pgpool_port0 = 9999other_wd_port0 = 9000relcache_expire = 0relcache_size = 256check_temp_table = oncheck_unlogged_table = onmemory_cache_enabled = offmemqcache_method = 'shmem'memqcache_memcached_host = 'localhost'memqcache_memcached_port = 11211memqcache_total_size = 67108864memqcache_max_num_cache = 1000000memqcache_expire = 0memqcache_auto_cache_invalidation = onmemqcache_maxcache = 409600memqcache_cache_block_size = 1048576memqcache_oiddir = '/var/log/pgpool/oiddir'white_memqcache_table_list = ''black_memqcache_table_list = '
五、两边都放好/usr/lib/postgresql/9.5/bin/failover_stream.sh数据库连接出问题后的脚本，给到postgres用户755权限
#! /bin/sh# Failover command for streaming replication.# This script assumes that DB node 0 is primary, and 1 is standby.## If standby goes down, do nothing. If primary goes down, create a# trigger file so that standby takes over primary node.## Arguments: $1: failed node id. $2: new master hostname. $3: path to# trigger file.failed_node=$1new_master=$2trigger_file=$3# Do nothing if standby goes down.if [ $failed_node = 1 ]; then exit 0;fi# Create the trigger file./usr/bin/ssh -T $new_master /bin/touch $trigger_fileexit 0;
六、启动前的准备
1、两边都新建配置文件中的目录
mkdir /var/run/pgpool #加入到/etc/rc.local，貌似这玩意儿新建以后会自动删掉还不自动建。。。mkdir /var/run/postgresql#加入到/etc/rc.localmkdir /var/log/pgpool
2、两边都copy个pgpool.conf出来
加入如下内容，否则会出现错误“psql: RROR: MD5 authentication is unsupported in replication, master-slave and parallel modes.HINT: check pg_hba.conf”
host all all 0.0.0.0/0 md5
3、pool_passwd中设置用户名密码验证
/usr/local/pgpool/bin/pg_md5 -m -p -u postgres pool_passwd
4、新建一个脚本专门用于停止pgpool的主服务的，当然也可以用作备机
vim /usr/local/etc/rm_pgpool.shps -ef | grep pgpool | cut -c 9-15 | xargs kill -9ps -ef | grep arping | cut -c 9-15 | xargs kill -9rm -rf /var/run/postgresql/.s.PGSQL.5432rm -rf /tmp/.s.PGPOOLWD_CMD.9000 /tmp/.s.PGSQL.9898ip addr del 192.168.180.221/24 dev eth0 label eth0:0
5、pgpool的自启动
vim /etc/rc.local/usr/local/etc/rm_pgpool.sh pgpool -f /usr/local/etc/pgpool.conf -F /usr/local/etc/pcp.conf -a /usr/local/etc/pool_hba.conf -D
七、测试
1、PG的主备切换
原理和我另外一篇PG流复制主备的一样，PGPOOL主要就是检测到比如主机PG挂掉了，调用failer_command在最新的主机即备机上创建一个文件从而激活备机。当然PGPOOL的VIP不会跟着漂移，这个可以在PG切换完成后再去切换
先打开主备机的pgpool\pg日志观察，防止某些错误没有看到导致切换出错
master:/etc/init.d/postgresql stop
slave:$PGDATA/recovery.conf变为recovery.done
slave:pg_controldata | grep cluster，显示为in production
slave:psql->show transaction_read_only，显示为off
slave:ifconfig，显示VIP在线
vip:show pool_nodes，显示原master主机状态为3且为standby、原slave备机状态为2且为primary
如果要原主机恢复为standby，则得把/tmp/trigger_file0删掉，重命名recovery.done为recovery.conf，然后再启动原主机
2、PGPOOL的主备切换
pgpool VIP :192.168.180.221 从 master飘到 slave，注意此时仅 pgpool 的 VIP 飘移， master,slave上的数据库角色不变(或者说是PG的主备关系不变), pgpool 的切换非常容易
关闭 pgpool ：
pgpool -m fast stop
启动pgpool并清除状态：
pgpool -f /usr/local/etc/pgpool.conf -F /usr/local/etc/pcp.conf -a /usr/local/etc/pool_hba.conf -D
八、后期操作
1、查看当前主备状态
使用pgpool登陆：show pool_nodes ;
0 192.168.180.222 5433 2 0.500000 primary 83
1 192.168.180.223 5433 2 0.500000 standby 02、如果当主节点挂掉再恢复为standby后，pool_nodes的status仍然为3，这时候就得两边pgpool都重启下，一般先重启pgpool的备机再重启主机即可，记住重启的时候肯定要一直Ping VIP，一定要在同一网段的其他机器上ping，非同一网段可能会被arp误导
九、错误
有时候关闭备机了，但是主机连pgpool -m fast stop都无法关闭，只能ps -ef | grep pgpool kill掉相关进程，还要netstat -anp | grep arping再kill掉这个进程，然后再去/var/run/postgresql和/tmp目录找到.s*打头的隐藏文件除了postgresql自己端口的基本上都是pgpool的删掉，否则启动报错。。。