Ovirt实现虚拟机通过NAT上网

环境说明

OS: CentOS Linux release 7.1.1503 (Core)
Ovirt-engine: ovirt-engine-3.5.3.1-1.el7
VDSM: vdsm-4.16.20-0.el7
GuestOS: CentOS release 6.5 (Final)
硬件说明：单网卡且只有一个IP：10.10.19.100（可连接外网）
附：此主机同时充当engine和node角色

1. 安装Centos7-mini并update（省略）

2. 安装和配置ovirt

初始化相关系统设置

  [root@localhost ~]# echo "ovirthost01.ctcnet.com" >/etc/hostname
  
  [root@localhost ~]# systemctl stop NetworkManager
  [root@localhost ~]# systemctl disable NetworkManager

  [root@ovirthost01 ~]# cat >/etc/sysconfig/network-scripts/ifcfg-p4p1 <<EOF
  > DEVICE=p4p1
  > TYPE=Ethernet
  > ONBOOT=yes
  > BOOTPROTO=static
  > IPADDR=10.10.19.100
  > PFEFIX=24
  > GATEWAY=10.10.19.254
  > DNS1=10.10.19.254
  > EOF

  [root@ovirthost01 ~]# echo "10.10.19.100 ovirthost01 ovirthost01.ctcnet.com" >>/etc/hosts

  [root@ovirthost01 ~]# echo "export PATH=/bin:/sbin:$PATH" >>~/.bashrc &&source .bashrc &&echo $PATH

  [root@ovirthost01 ~]# setenforce 0 &&sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config 
  
  [root@ovirthost01 ~]# systemctl stop firewalld && systemctl disable firewalld    
  
  [root@ovirthost01 ~]# systemctl stop iptables && systemctl disable iptables    
  
  [root@ovirthost01 ~]# yum install net-tools

  [root@ovirthost01 ~]# service postgresql initdb
  [root@ovirthost01 ~]# service postgresql start
  [root@ovirthost01 ~]# chkconfig postgresql on

安装和配置ovirt-engine

  [root@ovirthost01 ~]# yum localinstall -y http://resources.ovirt.org/pub/yum-repo/ovirt-release35.rpm 
  
  [root@ovirthost01 ~]# yum install -y ovirt-engine

  [root@ovirthost01 ~]# engine-setup

遇到如下错误：

  [ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
  [ INFO  ] Stage: Transaction commit
  [ INFO  ] Stage: Closing up
  [ INFO  ] Restarting nfs services
  [ ERROR ] Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute
  [ INFO  ] Stage: Clean up
            Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20150804004920-op260z.log
  [ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20150804005258-setup.conf'
  [ INFO  ] Stage: Pre-termination
  [ INFO  ] Stage: Termination
  [ ERROR ] Execution of setup failed

检查日志：

  less /var/log/ovirt-engine/setup/ovirt-engine-setup-20150804004920-op260z.log

日志中记录了如下错误（启动nfs-server失败）：

  2015-08-04 00:52:58 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:932 execute-output: ('/bin/systemctl', 'start', 'nfs-server.service') stdout:

  2015-08-04 00:52:58 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:937 execute-output: ('/bin/systemctl', 'start', 'nfs-server.service') stderr:
  Job for nfs-server.service failed. See 'systemctl status nfs-server.service' and 'journalctl -xn' for details.
  
  2015-08-04 00:52:58 DEBUG otopi.context context._executeMethod:152 method exception
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/otopi/context.py", line 142, in _executeMethod
      method['method']()
    File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/system/nfs.py", line 307, in _closeup
      state=state,
    File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 138, in state
      'start' if state else 'stop'
    File "/usr/share/otopi/plugins/otopi/services/systemd.py", line 77, in _executeServiceCommand
      raiseOnError=raiseOnError
    File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 942, in execute
      command=args[0],
  RuntimeError: Command '/bin/systemctl' failed to execute

根据日志提示检查下nfs-server服务状态(failed)：

  [root@ovirthost01 ~]# systemctl status nfs-server.service

  nfs-server.service - NFS server and services
     Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
     Active: failed (Result: exit-code) since Tue 2015-08-04 00:52:58 EDT; 7min ago
    Process: 21922 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=1/FAILURE)
    Process: 21919 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
   Main PID: 21922 (code=exited, status=1/FAILURE)
     CGroup: /system.slice/nfs-server.service
  
  Aug 04 00:52:58 ovirthost01.ctcnet.com rpc.nfsd[21922]: rpc.nfsd: writing fd to kernel failed: errno 111 (Connection refused)
  Aug 04 00:52:58 ovirthost01.ctcnet.com rpc.nfsd[21922]: rpc.nfsd: unable to set any sockets for nfsd
  Aug 04 00:52:58 ovirthost01.ctcnet.com systemd[1]: nfs-server.service: main process exited, code=exited, status=1/FAILURE
  Aug 04 00:52:58 ovirthost01.ctcnet.com systemd[1]: Failed to start NFS server and services.
  Aug 04 00:52:58 ovirthost01.ctcnet.com systemd[1]: Unit nfs-server.service entered failed state.

搜索下得知要先启动rpcbind:

  [root@ovirthost01 ~]# systemctl start rpcbind

再次启动nfs-server并检查其状态(success)：

  [root@ovirthost01 ~]# systemctl start  nfs-server.service

  [root@ovirthost01 ~]# systemctl status nfs-server.service

  nfs-server.service - NFS server and services
     Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled)
     Active: active (exited) since Tue 2015-08-04 01:10:26 EDT; 11s ago
    Process: 22042 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=0/SUCCESS)
    Process: 22040 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
   Main PID: 22042 (code=exited, status=0/SUCCESS)
     CGroup: /system.slice/nfs-server.service
  
  Aug 04 01:10:26 ovirthost01.ctcnet.com systemd[1]: Starting NFS server and services...
  Aug 04 01:10:26 ovirthost01.ctcnet.com systemd[1]: Started NFS server and services.

再次运行engine-setup成功：

  [ INFO  ] Cleaning stale zombie tasks and commands
   
    --== CONFIGURATION PREVIEW ==--
   
    Firewall manager                        : firewalld
    Update Firewall                         : True
    Host FQDN                               : ovirthost01.ctcnet.com
    Engine database name                    : engine
    Engine database secured connection      : False
    Engine database host                    : localhost
    Engine database user name               : engine
    Engine database host name validation    : False
    Engine database port                    : 5432
    Engine installation                     : True
    PKI organization                        : ctcnet.com
    NFS mount point                         : /data/iso
    Configure WebSocket Proxy               : True
    Engine Host FQDN                        : ovirthost01.ctcnet.com
   
    Please confirm installation settings (OK, Cancel) [OK]: 
  [ INFO  ] Cleaning async tasks and compensations
  [ INFO  ] Checking the Engine database consistency
  [ INFO  ] Stage: Transaction setup
  [ INFO  ] Stopping engine service
  [ INFO  ] Stopping ovirt-fence-kdump-listener service
  [ INFO  ] Stopping websocket-proxy service
  [ INFO  ] Stage: Misc configuration
  [ INFO  ] Stage: Package installation
  [ INFO  ] Stage: Misc configuration
  [ INFO  ] Backing up database localhost:engine to '/var/lib/ovirt-engine/backups/engine-20150804012706.WAg6oL.dump'.
  [ INFO  ] Creating/refreshing Engine database schema
  [ INFO  ] Configuring WebSocket Proxy
  [ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
  [ INFO  ] Stage: Transaction commit
  [ INFO  ] Stage: Closing up
           
            --== SUMMARY ==--
           
  [WARNING] Less than 16384MB of memory is available
            SSH fingerprint: F6:83:42:C9:FF:0F:A3:CE:ED:F5:85:EC:27:22:5F:E7
            Internal CA A3:CF:B5:C1:B4:29:8B:36:A5:9B:EB:69:99:A6:8D:5B:55:81:36:8F
            Web access is enabled at:
                http://ovirthost01.ctcnet.com:80/ovirt-engine
                https://ovirthost01.ctcnet.com:443/ovirt-engine
           
            --== END OF SUMMARY ==--
           
  [ INFO  ] Starting engine service
  [ INFO  ] Restarting httpd
  [ INFO  ] Stage: Clean up
            Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20150804012601-6zut1o.log
  [ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20150804012750-setup.conf'
  [ INFO  ] Stage: Pre-termination
  [ INFO  ] Stage: Termination
  [ INFO  ] Execution of setup completed successfully

备份ovirt-engine文件和数据库

  [root@ovirthost01 ~]# engine-backup --mode=backup --file=engine_backup00 --log=./engine_backup00.log

如果想了解具体包含哪些配置文件，可以解压engine_bakcup00来查看：

  [root@ovirthost01 ~]# tar -jxvf engine_backup00

添加数据中心，集群，主机，数据存储域和ISO域（省略）

    Item              Name                 Path/IP
    -------------------------------------------------
    DataCenter        ctc_dc                    ——
    Cluster           ctc_cluster01            ——
    Host                host01                10.10.19.100
    Data_Stor_Dom     data_stor            10.10.19.99:/volume1/nfsstor
    ISO_Stor_Dom      iso_stor            10.10.19.100:/data/iso

注意：由于我采用vdsm和engine都在同一主机上，添加此主机到集群（安装vdsm）后会自动启动iptables,导致无法访问webadmin-portal（port:443/80）。添加相关条目到/etc/sysconfig/iptables来解决,内容如下：

  [root@ovirthost01 ~]# cat /etc/sysconfig/iptables

  # Generated by iptables-save v1.4.21 on Tue Aug  4 02:58:26 2015
  *filter
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [250:63392]
  -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
  -A INPUT -p icmp -j ACCEPT
  -A INPUT -i lo -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 54321 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 111 -j ACCEPT
  -A INPUT -p udp -m udp --dport 111 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
  -A INPUT -p udp -m udp --dport 161 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 16514 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
  -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
  -A INPUT -p tcp -m multiport --dports 5900:6923 -j ACCEPT
  -A INPUT -p tcp -m multiport --dports 49152:49216 -j ACCEPT
  -A INPUT -j REJECT --reject-with icmp-host-prohibited
  -A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited
  COMMIT
  # Completed on Tue Aug  4 02:58:26 2015

  [root@ovirthost01 ~]# systemctl restart iptables

3. 创建虚拟机（通过NAT连接外网）

1.导入iso镜像文件

    [root@ovirthost01 ~]# engine-iso-uploader -i iso_stor upload  /data/iso/*.iso

    Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort): 
    Uploading, please wait...
    INFO: Start uploading /data/iso/CentOS-6.5-x86_64-minimal.iso 
    INFO: /data/iso/CentOS-6.5-x86_64-minimal.iso uploaded successfully
    INFO: Start uploading /data/iso/CentOS-7.0-1406-x86_64-Minimal.iso 
    INFO: /data/iso/CentOS-7.0-1406-x86_64-Minimal.iso uploaded successfully
    INFO: Start uploading /data/iso/virtio-win-0.1.96.iso 
    INFO: /data/iso/virtio-win-0.1.96.iso uploaded successfully

2.创建NAT网络配置文件/etc/libvirt/qemu/networks/nat.xml,内容如下

    <network>
        <name>nat</name>
        <uuid>b09d09a8-ebbd-476d-9045-e66012c9e83d</uuid>
        <forward mode='nat'/>
        <bridge name='natbr0' stp='on' delay='0' />
        <mac address='52:54:00:9D:82:DE'/>
        <ip address='192.168.1.1' netmask='255.255.255.0'>
            <dhcp>
                <range start='192.168.1.2' end='192.168.1.250' />
            </dhcp>
        </ip>
    </network>

3.通过libvirt/virsh创建NAT网络

    [root@ovirthost01 ~]# cat /etc/pki/vdsm/keys/libvirt_password 
    shibboleth
    
    [root@ovirthost01 ~]# virsh

    Welcome to virsh, the virtualization interactive terminal.
    
    Type:  'help' for help with commands
           'quit' to quit
    
    virsh # connect qemu:///system
    Please enter your authentication name: vdsm@ovirt
    Please enter your password: shibboleth
    
    virsh # net-list
     Name                 State      Autostart     Persistent
    ----------------------------------------------------------
     ;vdsmdummy;          active     no            no
     vdsm-ovirtmgmt       active     yes           yes
    
    virsh # net-define /etc/libvirt/qemu/networks/nat.xml
    Network nat defined from /etc/libvirt/qemu/networks/nat.xml
    
    virsh # net-autostart nat
    Network nat marked as autostarted
    
    virsh # net-start nat
    Network nat started
    
    virsh # net-list --all
     Name                 State      Autostart     Persistent
    ----------------------------------------------------------
     ;vdsmdummy;          active     no            no
     nat                  active     yes           yes
     vdsm-ovirtmgmt       active     yes           yes

    以上操作将创建nat功能的网桥，如下
    [root@ovirthost01 ~]# brctl show
    bridge name     bridge id               STP enabled     interfaces
    ;vdsmdummy;             8000.000000000000       no
    natbr0          8000.5254009d82de       yes             natbr0-nic
    ovirtmgmt               8000.b083fea27fed       no              p4p1

4.安装vdsm-hook-extnet

    [root@ovirthost01 ~]# yum install -y vdsm-hook-extnet

注：此处将下载extnet的hooks文件并存放到以下两目录

    [root@ovirthost01 ~]# ll /usr/libexec/vdsm/hooks/before_device_create
    total 4
    -rwxr-xr-x. 1 root root 1925 Jun  5 01:47 50_extnet
    [root@ovirthost01 ~]# ll /usr/libexec/vdsm/hooks/before_nic_hotplug
    total 4
    -rwxr-xr-x. 1 root root 1925 Jun  5 01:47 50_extnet

5.添加自定义设备属性extnet

    [root@ovirthost01 ~]# engine-config -s CustomDeviceProperties='{type=interface;prop={extnet=^[a-zA-Z0-9_ ---]+$}}'
    Please select a version:
    1. 3.0
    2. 3.1
    3. 3.2
    4. 3.3
    5. 3.4
    6. 3.5
    6

    [root@ovirthost01 ~]# engine-config -g CustomDeviceProperties
    CustomDeviceProperties:  version: 3.0
    CustomDeviceProperties:  version: 3.1
    CustomDeviceProperties:  version: 3.2
    CustomDeviceProperties:  version: 3.3
    CustomDeviceProperties: {type=interface;prop={SecurityGroups=^(?:(?:[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}, *)*[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12}|)$}} version: 3.4
    CustomDeviceProperties: {type=interface;prop={extnet=^[a-zA-Z0-9_ ---]+$}} version: 3.5

    [root@ovirthost01 ~]# systemctl restart ovirt-engine

6.通过webadmin-portal创建虚拟机,并通过ISO安装GuestOS(省略)

7.添加nat端口配置集

图1-ovirt-nat0.JPG

8.添加vnic到虚拟机，并关联nat端口配置集

ovirt-nat1.JPG

9.进入到虚拟机验证（成功）

检查网卡是否添加

    [root@VM01-CentOS6 ~]# ifconfig -a

图3-ovirt-nat2.JPG

从vdsm主机的dhcp服务器处获取IP

    [root@VM01-CentOS6 ~]# dhclient eth0

图4-ovirt-nat3.JPG

通过ping外网来检查NAT是否成功

    [root@VM01-CentOS6 ~]# ping www.ovirt.org

图5-ovirt-nat4.JPG

10.参考资料

http://www.ovirt.org/VDSM-Hooks/network-nat
http://blog.lofyer.org/add-nat-ovirt-vdsm-hooks/
http://users.ovirt.narkive.com/WVp1moNk/ovirt-users-ovirt-3-5-nat
https://access.redhat.com/documentation/zh-CN/Red_Hat_Enterprise_Virtualization/3.5/html-single/Installation_Guide/index.html

最后编辑于：2017.11.27 03:16:52

人面猴
序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
沈念sama阅读 194,242评论 5赞 459
死咒
序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
沈念sama阅读 81,769评论 2赞 371
救了他两次的神仙让他今天三更去死
文/潘晓璐我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
开封第一讲书人阅读 141,484评论 0赞 319
道士缉凶录：失踪的卖姜人
文/不坏的土叔我叫张陵，是天一观的道长。经常有香客问我，道长，这世上最难降的妖魔是什么？我笑而不...
开封第一讲书人阅读 52,133评论 1赞 263
港岛之恋（遗憾婚礼）
正文为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
茶点故事阅读 61,007评论 4赞 355
恶毒庶女顶嫁案：这布局不是一般人想出来的
文/花漫我一把揭开白布。她就那样静静地躺着，像睡着了一般。火红的嫁衣衬着肌肤如雪。梳的纹丝不乱的头发上，一...
开封第一讲书人阅读 46,080评论 1赞 272
城市分裂传说
那天，我揣着相机与录音，去河边找鬼。笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。我是一名探鬼主播，决...
沈念sama阅读 36,496评论 3赞 381
双鸳鸯连环套：你想象不到人心有多黑
文/苍兰香墨我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
开封第一讲书人阅读 35,190评论 0赞 253
万荣杀人案实录
序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
沈念sama阅读 39,464评论 1赞 290
护林员之死
正文独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋以下内容为张勋视角年9月15日...
茶点故事阅读 34,549评论 2赞 309
白月光启示录
正文我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
茶点故事阅读 36,330评论 1赞 326
活死人
序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
沈念sama阅读 32,205评论 3赞 312
日本核电站爆炸内幕
正文年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
茶点故事阅读 37,567评论 3赞 298
男人毒药：我在死后第九天来索命
文/蒙蒙一、第九天我趴在偏房一处隐蔽的房顶上张望。院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
开封第一讲书人阅读 28,889评论 0赞 17
一桩弑父案，背后竟有这般阴谋
文/苍兰香墨我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。一阵脚步声响...
开封第一讲书人阅读 30,160评论 1赞 250
情欲美人皮
我被黑心中介骗来泰国打工，没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。一个月前我还...
沈念sama阅读 41,475评论 2赞 341
代替公主和亲
正文我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
茶点故事阅读 40,650评论 2赞 335

Ovirt实现虚拟机通过NAT上网

环境说明

1. 安装Centos7-mini并update（省略）

2. 安装和配置ovirt

3. 创建虚拟机（通过NAT连接外网）

推荐阅读更多精彩内容