系统环境：
VMware Workstation14.5 +linux 7.6(64位)+Oracle 19.3.0.0.0
两个节点mnt1(节点1)和mnt2(节点2)
hosts：
vi /etc/hosts
192.168.174.141 cat1
192.168.174.142 cat2
192.168.174.143 cat1vip
192.168.174.144 cat2vip
192.168.174.145 catscanip
192.168.160.141 cat1priv
192.168.160.142 cat2priv
节点1：
hostname:cat1 ip:192.168.174.141 scanip:192.168.174.145 catscanip dbname=catdb instance_name=catdb1
节点2：
hostname:cat1 ip:192.168.174.142 scanip:192.168.174.145 catscanip dbname=catdb instance_name=catdb2
0.安装部署前检查，详情见19_rac excel表。
1.检查网络，有两个不同的网段,两个静态IP.
ifconfig
网卡1（public ip）：vi /etc/sysconfig/network-scripts/ifcfg-ens33
BOOTPROTO=static
IPADDR=192.168.0.116（填你的ip）
NETMASK=255.255.255.0 （填你的掩码值）
GATEWAY=192.168.0.1 (默认网关)
网卡2（心跳ip）：vi /etc/sysconfig/network-scripts/ifcfg-ens34
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=10.0.0.101
NETMASK=255.255.255.0
systemctl restart network //重启网络
2.检查linux系统版本,需要在linux7.4版本以上,机器的系统要同样的。
cat /etc/redhat-release
uname -r
3.检查安装所需的磁盘
/tmp目录大小至少：1GB
安装Grid Infrastracture所需空间：12GB
安装Oracle Database所需空间：7.3GB
此外安装过程中分析、收集、跟踪文件所需空间：10GB
建议总共至少100GB（此处不包含ASM或NFS的空间需求）
# df -h
4.调整好时区和时间
date
rm -rf /etc/localtime
ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
5.检查iso镜像是否挂上。
mount /dev/sr0 /mnt
6.内存检查
内存大小：至少8GB
Swap大小：
当内存为4GB-16GB时，Swap需要大于等于系统内存。
当内存大于16GB时，Swap等于16GB即可。
# grep MemTotal /proc/meminfo
# grep SwapTotal /proc/meminfo
1、安装软件包
mount /dev/sr0 /mnt
cd /etc/yum.repos.d/
rm -rf *
vi /etc/yum.repos.d/ol.repo
[local-source]
name=ol
baseurl=file:///mnt/
ebabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
安装rpm依赖包
yum -y install bc gcc gcc-c++ binutils make gdb cmake glibc ksh elfutils-libelf elfutils-libelf-devel fontconfig-devel glibc-devel libaio libaio-devel libXrender libXrender-devel libX11 libXau sysstat libXi libXtst libgcc librdmacm-devel libstdc++ libstdc++-devel libxcb net-tools nfs-utils compat-libcap1 compat-libstdc++ smartmontools targetcli python python-configshell python-rtslib python-six unixODBC unixODBC-devel
yum install -y xterm
linux7缺少该包，需要去网上下载手动安装，或者不安装不会影响数据库的使用，详情看mos文档2254198.1，Missing or Ignored package compat-libstdc++-33-3.2.3 causes Text Issues (文档 ID 2254198.1)
ls -l compat* elfutils* | awk -v rpmpackage="" '{rpmpackage=$NF" "rpmpackage}END{print "yum -y localinstall "rpmpackage}' | bash
无安装桌面：
安装桌面组件包 #yum -y groupinstall "Server with GUI" \\ 安装图形化程序
启动桌面——start
2、创建用户、组、目录
groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper
groupadd -g 54324 backupdba
groupadd -g 54325 dgdba
groupadd -g 54326 kmdba
groupadd -g 54327 asmdba
groupadd -g 54328 asmoper
groupadd -g 54329 asmadmin
groupadd -g 54330 racdba
useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle
useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid
passwd oracle
passwd grid
mkdir -p /u01/app/19.3.0/grid
mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/product/19.3.0/db_1
chown -R grid:oinstall /u01
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/
mkdir -p /u01/app/oraInventory
chown -R grid:oinstall /u01/app/oraInventory
chmod -R 775 /u01/app/oraInventory
作用 路径
grid_home /u01/app/19.3.0/grid
grid_base /u01/app/grid
oracle_base /u01/app/oracle
Oracle_home /u01/app/oracle/product/19.3.0/db_1
3、配置共享磁盘（多路径+udev）
配置multipath加udev
建立共享磁盘（虚拟机）
切换到vmware安装目录 cd D:\VMware\vmware
cmd 命令
.\vmware-vdiskmanager.exe -c -s 10G -a lsilogic -t 2 "D:\VMware\vmware\sharedisks\19c\OCR.vmdk"
.\vmware-vdiskmanager.exe -c -s 40G -a lsilogic -t 2 "D:\VMware\vmware\sharedisks\19c\DATA.vmdk"
.\vmware-vdiskmanager.exe -c -s 10G -a lsilogic -t 2 "D:\VMware\vmware\sharedisks\19c\ARCH.vmdk"
分别打开两台虚拟机目录中的vmx文件，在最后一行添加：
scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedBus = "virtual"
scsi1:1.present = "TRUE"
scsi1:1.mode = "independent-persistent"
scsi1:1.filename = "D:\VMware\vmware\sharedisks\19c\OCR.vmdk"
scsi1:1.deviceType = "Disk"
scsi1:2.present = "TRUE"
scsi1:2.mode = "independent-persistent"
scsi1:2.filename = "D:\VMware\vmware\sharedisks\19c\DATA.vmdk"
scsi1:2.deviceType = "Disk"
scsi1:3.present = "TRUE"
scsi1:3.mode = "independent-persistent"
scsi1:3.filename = "D:\VMware\vmware\sharedisks\19c\ARCH.vmdk"
scsi1:3.deviceType = "Disk"
disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
1.双节点安装multipath的包,先后顺序不能乱：
rpm -ivh device-mapper-multipath-libs-0.4.9-123.el7.x86_64.rpm
rpm -ivh device-mapper-multipath-0.4.9-123.el7.x86_64.rpm
2.启动multipath（rac1、rac2都跑）
modprobe dm-multipath
modprobe dm-round-robin
3.开机自启动
systemctl enable multipathd.service
systemctl list-unit-files | grep multi
multipathd.service enabled
multi-user.target static
4.启动multipath服务：
service multipathd start
5.多路径配置（rac、rac2）
oracle RAC所需多路径配置要点是一个RAC节点一个RAC节点一次执行如下操作（如果需要再添加使用多路径的存储，也需遵循如下步骤）：
service multipathd stop
multipath -F
获取wwid：
在 Linux 7下，可以使用如下命令：
# /usr/lib/udev/scsi_id -g -u /dev/sdb 360050768018187d15000000000000039
/usr/lib/udev/scsi_id -g -u /dev/sdc 360050768018187d1500000000000003a
/usr/lib/udev/scsi_id -g -u /dev/sdd 360050768018187d15000000000000038
先查看是否存在/etc/multipath.conf，不存在
方法1:创建配置文件：cp /etc/multipath_bak.conf /etc/multipath.conf
vi /etc/multipath.conf
defaults {
user_friendly_names yes
}
devices {
device {
vendor "(COMPAQ|HP)"
product "HSV1[01]1|HSV2[01]0|HSV300|HSV4[05]0"
#getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
features "0"
hardware_handler "0"
path_selector "round-robin 0"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
no_path_retry 12
rr_min_io 1000
path_checker tur
prio alua
}
}
multipaths {
multipath {
wwid 360050768018187d15000000000000039
alias ARCH
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 360050768018187d1500000000000003a
alias OCR
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
multipath {
wwid 360050768018187d15000000000000038
alias DATA
path_grouping_policy multibus
path_selector "round-robin 0"
failback immediate
rr_weight priorities
no_path_retry queue
}
}
blacklist {
devnode "^(sda|ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^asm/*" # Here
devnode "ofsctl" # Here
}
#启动多路径：
service multipathd restart
启动multipath配置后，会在/dev/mapper下生成多路径逻辑盘
cd /dev/mapper
ll
#查看多路径的状态：multipath -ll
---验证权限
ls -l /dev/dm-*
---验证命名结果
ls -l /dev/mapper/*
udev步骤：
1.查看磁盘UUID
lsscsi -i
2.编辑 UDEV Rules File
vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-360050768018187d15000000000000039",SYMLINK+="asm_ARCH",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-360050768018187d1500000000000003a",SYMLINK+="asm_OCR",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-360050768018187d15000000000000038",SYMLINK+="asm_DATA",OWNER="grid",GROUP="asmadmin",MODE="0660"
3.scp rules文件至其他节点
scp /etc/udev/rules.d/99-oracle-asmdevices.rules node2:/etc/udev/rules.d/
4. 重新加载rules文件
udevadm control --reload-rules
udevadm trigger
5.查看绑定后设备信息
[root@test1 rules.d]# ls -l /dev/asm*
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test1 -> dm-2
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test2 -> dm-3
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test3 -> dm-4
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test4 -> dm-5
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test5 -> dm-6
lrwxrwxrwx 1 root root 4 Aug 20 14:42 /dev/asm_test6 -> dm-7
[root@test1 rules.d]# ls -l /dev/dm-*
brw-rw---- 1 grid asmadmin 253, 2 Aug 20 14:42 /dev/dm-2
brw-rw---- 1 grid asmadmin 253, 3 Aug 20 14:42 /dev/dm-3
brw-rw---- 1 grid asmadmin 253, 4 Aug 20 14:42 /dev/dm-4
brw-rw---- 1 grid asmadmin 253, 5 Aug 20 14:42 /dev/dm-5
brw-rw---- 1 grid asmadmin 253, 6 Aug 20 14:42 /dev/dm-6
brw-rw---- 1 grid asmadmin 253, 7 Aug 20 14:42 /dev/dm-7
4、配置系统参数
4.1修改主机名
vi /etc/hosts
192.168.174.141 cat1
192.168.174.142 cat2
192.168.174.143 cat1vip
192.168.174.144 cat2vip
192.168.174.145 catscanip
192.168.160.141 cat1priv
192.168.160.142 cat2priv
vi /etc/hostname
cat1 cat2
hostname cat1
hostname cat2
4.2配置系统文件
--/etc/security/limits.conf 是 Linux 资源使用配置文件，用来限制用户对系统资源的使用
--https://www.cnblogs.com/pzk7788/p/7250723.html
vi /etc/security/limits.conf
添加
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
grid soft nproc 16384
grid hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
oracle soft nproc 16384
oracle hard nproc 16384
root soft nproc 2047
vi /etc/pam.d/login 添加：
session required pam_limits.so
设置内核参数，编辑文件并使之生效，其值不小于如下设置。
以下是参考内容
vi /etc/sysctl.conf
#for oracle 19c rac
fs.aio-max-nr = 4194304
fs.file-max = 6815744
####kernel setting
kernel.shmall = 4194304
kernel.shmmax = 16106127360
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
kernel.panic_on_oops = 1
kernel.panic = 10
#### Net Setting
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 4194304
####Memory Setting
vm.vfs_cache_pressure=200
vm.swappiness=10
vm.min_free_kbytes=102400
#vm.nr_hugepages=10
配置生效，在全部节点操作
sysctl -p
vi /etc/profile添加：
if [ $USER = "oracle" ] || [ $USER = "grid" ] ; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -u 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
4.3禁用SELinux、防火墙、NTP、chrony服务、avahi-daemon
SELinux建议禁用此功能。
vi /etc/selinux/config
SELINUX=disabled
需要重启。
防火墙禁用：
systemctl disable firewalld
systemctl stop firewalld
systemctl stop firewalld.service
systemctl disable firewalld.service
验证：
systemctl list-unit-files | grep fire
关闭NTP:
systemctl stop ntpd.service
systemctl disable ntpd.service
mv /etc/ntp.conf /etc/ntp.conf.orig
systemctl status ntpd
● ntpd.service - Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled)
Active: inactive (dead)
关闭chrony服务，移除chrony配置文件。
systemctl list-unit-files|grep chronyd
systemctl status chronyd
systemctl disable chronyd
systemctl stop chronyd
mv /etc/chrony.conf /etc/chrony.conf_bak
关闭avahi-daemon
systemctl stop avahi-daemon
systemctl disable avahi-daemon
4.4 oracle、grid用户环境变量配置
oracle用户，2个节点上配置：
vi ~/.bash_profile 编辑后source ~/.bash_profile
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=test1 #节点2改为test2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/19.3.0/db_1
export TNS_ADMIN=$ORACLE_HOME/network/admin
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
export NLS_LANG=AMERICAN_AMERICA.UTF8
grid用户，2个节点上配置：
vi ~/.bash_profile 编辑后source ~/.bash_profile
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_SID=+ASM1 # RAC1
#export ORACLE_SID=+ASM2 # RAC2
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/19.3.0/grid
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
umask 022
export NLS_LANG=AMERICAN_AMERICA.UTF8
4.5 配置zeroconf
vi /etc/sysconfig/network
NOZEROCONF=yes
4.6 禁用透明大页（Transparent HugePages）
cat /sys/kernel/mm/transparent_hugepage/enabled //查看状态
vi /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rhgb quiet transparent_hugepage=never"
GRUB_DISABLE_RECOVERY="true"
执行生效命令 grub2-mkconfig -o /boot/grub2/grub.cfg
重启OS后确认是否生效
[root@test12c1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
4.7 设置Disk I/O Scheduler deadline
为了获得Oracle ASM的最佳性能，Oracle建议您使用Deadline I / O Scheduler
磁盘I / O调度程序对磁盘I / O的请求进行重新排序，延迟或合并，以实现更高的吞吐量和更低的延迟。Linux有多个磁盘I / O调度程序可用，包括Deadline，Noop，Anticipatory和Completely Fair Queuing（CFQ）。
vi /etc/udev/rules.d/60-oracle-schedulers.rules
添加如下内容
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"
运行生效
udevadm control --reload-rules
4.8 配置/dev/shm大小
不配置会报错ORA-00845,简单老说就是MEMORY_MAX_TARGET的设置不能超过 /dev/shm的大小。
vi /etc/fstab
tmpfs /dev/shm tmpfs defaults,size=2G 0 0
重新挂载：
mount -o remount /dev/shm
验证： df -h
4.9 关闭THP及NUMA(类似关闭透明大页操作，可略过。)
vi /etc/default/grub
在GRUB_CMDLINE_LINUX添加下列选项，用于关闭THP和NUMA
transparent_hugepage=never numa=off
编译并重启主机
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg