nginx+docker怎么实现paas
这篇文章主要介绍“nginx+docker怎么实现paas”,在日常操作中,相信很多人在nginx+docker怎么实现paas问题上存在疑惑,小编查阅了各式资料,整理出简单好用的操作方法,希望对大家解答”nginx+docker怎么实现paas”的疑惑有所帮助!接下来,请跟着小编一起来学习吧!
创新互联 - 联通机房服务器托管,四川服务器租用,成都服务器租用,四川网通托管,绵阳服务器托管,德阳服务器托管,遂宁服务器托管,绵阳服务器托管,四川云主机,成都云主机,西南云主机,联通机房服务器托管,西南服务器托管,四川/成都大带宽,成都机柜租用,四川老牌IDC服务商
前置
更新了安装说明。集中化了全局变量,注意deps prepare部分,bridge-utils是为了控制cni制造的那个openfaas0虚拟网卡用。安装docker.io,是ubuntu上它可以同时安装containerd1.3.3和runc
从Docker 1.11开始,Docker容器运行已经不是简单的通过Docker daemon来启动,而是集成了containerd、runC等多个组件。如果去搜索一番,就会发现:docker-containerd 就是 containerd,而 docker-runc 就是 runc。containerd是真正管控容器的daemon,执行容器的时候用的是runc。 为什么 要分的七零八散呢?为了防止docker一家独大,docker当年的实现被拆分出了几个标准化的模块,,标准化的目的是模块是可被其他实现替换的,其实也是为了实现类llvm的组件化可开发效果(软件抽象上,源头如果有分才能合,如果一开始就是合的就难分)。也是为了分布式效果。docker也像git一样做分布式部件化了,分布式就是设置2个部件,cliserver,这样在本地和远程都可这样架构。
而为什么是dockerio而不是docker-ce: 事实是我还发现,有些系统上,安装了docker-ce再安装containerd。会导致系统出问题Cannot connect to the Docker daemon at 。docker与containerd不兼容,所以只好安装ubuntu维护的docker.io这种解决了containerd依赖的,它默认依赖containerd和runc(不过稍后我们会提到替换升级containerd的版本)。我们选用加入了最新cn ubuntu deb src后apt-get update得到的sudo apt install docker.io=19.03.6-0ubuntu1~18.04.1,sudo apt-cache madison docker.io出来的版本.
#!/bin/bash ## currently tested under ubuntu1804 64b,easy to be ported to centos(can be tested with replacing apt-get and /etc/systemd/system) ## How to use this script in a cloudhost ## su root and then: ./panel.sh -d 'your domain to be binded' -m 'email you use to pass to certbot' -p 'your inital passwords'(email and passwords are not neccessary,feed email only if you encount the "toomanyrequestofagivetype" error) ## (no prefix https/http needed,should bind to the right ip ahead for laster certbot working) export DOMAIN_NAME='' export EMAIL_NAME='some@some.com' export PANEL_TYPE='0' export PASS_INIT='5cTWUsD75ZgL3VJHdzpHLfcvJyOrUnza1jr6KXry5pXUUNmGtqmCZU4yGoc9yW4' MIRROR_PATH="http://default-8g95m46n2bd18f80.service.tcloudbase.com/d/demos" # the pai backend SERVER_PATH=${MIRROR_PATH}/pai/pai-agent/stable/pai_agent_framework PAI_MATE_SERVER_PATH=${MIRROR_PATH}/pai/pai-mate/stable/install # the openfaas backend OPENFAAS_PATH=${MIRROR_PATH}/faasd # the code-server web ide CODE_SERVER_PATH=${MIRROR_PATH}/codeserver #install dir INSTALL_DIR="/root/.local" CONFIG_DIR="/root/.config" # datadir only for pai and common data DATA_DIR="/data" while [[ $# -ge 1 ]]; do case $1 in -d|--domain) shift DOMAIN_NAME="$1" shift ;; -m|--mail) shift EMAIL_NAME="$1" shift ;; -t|--paneltype) shift PANEL_TYPE="$1" shift ;; -p|--passinit) shift PASS_INIT="$1" shift ;; *) if [[ "$1" != 'error' ]]; then echo -ne "\nInvaild option: '$1'\n\n"; fi echo -ne " Usage(args are self explained):\n\tbash $(basename $0)\t-d/--domain\n\t\t\t\t\-m/--mail\n\t\t\t\t\-t/--paneltype\n\t\t\t\t-p/--passinit\n\t\t\t\t\n" exit 1; ;; esac done [[ "$EUID" -ne '0' ]] && echo "Error:This script must be run as root!" && exit 1; beginTime=$(date +%s) # write log with time writeProgressLog() { echo "[`date '+%Y-%m-%d %H:%M:%S'`][$1][$2]" echo "[`date '+%Y-%m-%d %H:%M:%S'`][$1][$2]" >> ${DATA_DIR}/h6/access.log } # update install progress updateProgress() { progress=$1 message=$2 status=$3 installType=$4 # echo "=====================$installType progress=======================" echo "=======================$installType progress=======================" >> ${DATA_DIR}/h6/access.log writeProgressLog "installType" $installType writeProgressLog "progress" $progress writeProgressLog "status" $status echo $message >> ${DATA_DIR}/h6/access.log if [ $status == "0" ]; then code=0 message="success" else code=1 message="$installType error" # exit 1 fi cat << EOF > ${DATA_DIR}/h6/progress.json { "code": $code, "message": "$message", "data": { "installType": "$installType", "progress": $progress } } EOF if [ $status == "0" ]; then code=0 message="success" else code=1 message="$installType error" # exit 1 fi if [ $status != "0" ]; then echo $message >> ${DATA_DIR}/h6/installErr.log fi } echo "=====================begin .....=====================" echo "PANEL_TYPE: ${PANEL_TYPE}" echo "DOMAIN_NAME: ${DOMAIN_NAME}" echo "SERVER_PATH: ${MIRROR_PATH}" echo "OPENFAAS_PATH: ${OPENFAAS_PATH}" echo "PAI_MATE_SERVER_PATH: ${PAI_MATE_SERVER_PATH}" echo "CODE_SERVER_PATH: ${CODE_SERVER_PATH}" echo "INSTALL_DIR: ${INSTALL_DIR}" rm -rf ${DATA_DIR}/h6 mkdir -p ${DATA_DIR}/h6 rm -rf ${DATA_DIR}/h6/index.json rm -rf ${DATA_DIR}/logs mkdir -p ${DATA_DIR}/logs mkdir -p ${INSTALL_DIR}/bin mkdir -p ${CONFIG_DIR} echo "=====================deps prepare progress(this may take long...)=======================" msg=$( #begin if [ $PANEL_TYPE == "0" ]; then apt-key adv --recv-keys --keyserver keyserver.Ubuntu.com 3B4FE6ACC0B21F32 echo deb http://cn.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse >> /etc/apt/sources.list echo deb http://cn.archive.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse >> /etc/apt/sources.list echo deb http://cn.archive.ubuntu.com/ubuntu/ bionic-updates main restricted universe multiverse >> /etc/apt/sources.list echo deb http://cn.archive.ubuntu.com/ubuntu/ bionic-proposed main restricted universe multiverse >> /etc/apt/sources.list echo deb http://cn.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse >> /etc/apt/sources.list apt-get update apt-get install docker.io=19.03.6-0ubuntu1~18.04.1 --no-install-recommends bridge-utils -y apt-get install nginx python git python-certbot-nginx -y # sed '1{:a;N;5!b a};$d;N;P;D' -i /etc/apt/sources.list # apt-get update else apt-get update && apt-get install git nginx gcc python3.6 python3-pip python3-virtualenv python-certbot-nginx golang -y fi 2>&1) status=$? updateProgress 30 "$msg" "$status" "deps prepare"
基础组件代码:nginx front and docker backend
这部分虽然写死了各条转发。但重点在于如何根据具体的转发需要布置代码。这里的理论在于:如果代理服务器地址(proxy_pass后面那个)中是带有URI的,此URI会替换掉 location 所匹配的URI部分。 而如果代理服务器地址中是不带有URI的,则会用完整的请求URL来转发到代理服务器。
confignginx() { echo "=====================certbot renew+start+init progress=======================" systemctl enable nginx.service systemctl start nginx #cp -f /lib/systemd/system/certbot.service /etc/systemd/system/certbot-renew.service #echo '[Install]' >> /etc/systemd/system/certbot-renew.service #echo 'WantedBy=multi-user.target' >> /etc/systemd/system/certbot-renew.service #cp -f /lib/systemd/system/certbot.timer /etc/systemd/system/certbot-renew.timer # sed -i "s/renew/renew --nginx/g" /etc/systemd/system/certbot-renew.service rm -rf /etc/systemd/system/certbot-renew.service cat << 'EOF' > /etc/systemd/system/certbot-renew.service [Unit] Description=Certbot Documentation=file:///usr/share/doc/python-certbot-doc/html/index.html Documentation=https://letsencrypt.readthedocs.io/en/latest/ [Service] Type=oneshot ExecStart=/usr/bin/certbot -q renew PrivateTmp=true [Install] WantedBy=multi-user.target EOF rm -rf /etc/systemd/system/certbot-renew.timer cat << 'EOF' > /etc/systemd/system/certbot-renew.timer [Unit] Description=Run certbot twice daily [Timer] OnCalendar=*-*-* 00,12:00:00 RandomizedDelaySec=43200 Persistent=true [Install] WantedBy=timers.target EOF msg=$( #first time renew certbot certonly --quiet --standalone --agree-tos --non-interactive -m ${EMAIL_NAME} -d ${DOMAIN_NAME} --pre-hook "systemctl stop nginx" systemctl daemon-reload systemctl enable certbot-renew.service systemctl start certbot-renew.service systemctl start certbot-renrew.timer 2>&1) status=$? updateProgress 40 "$msg" "$status" "certbot renew+start+init" echo "=====================nginx reconfig progress=======================" # add nginx conf rm -rf /etc/nginx/conf.d/default.conf cat << 'EOF' > /etc/nginx/conf.d/default.conf server { listen 443 http2 ssl; listen [::]:443 http2 ssl; server_name DOMAIN_NAME; ssl on; ssl_certificate /etc/letsencrypt/live/DOMAIN_NAME/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/DOMAIN_NAME/privkey.pem; ssl_session_timeout 5m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE; ssl_prefer_server_ciphers on; location / { proxy_pass http://localhost:PORT; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection upgrade; proxy_set_header Accept-Encoding gzip; } location /pai/ { proxy_pass http://localhost:5523; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection upgrade; proxy_set_header Accept-Encoding gzip; } location /faasd/ { proxy_pass http://localhost:8080/ui/; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection upgrade; proxy_set_header Accept-Encoding gzip; } location /codeserver/ { proxy_pass http://localhost:5000/; proxy_redirect http:// https://; proxy_set_header Host $host:443/codeserver; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection upgrade; proxy_set_header Accept-Encoding gzip; } } server { listen 80; server_name DOMAIN_NAME; if ($host = DOMAIN_NAME) { return 301 https://$host$request_uri; } return 404; } EOF sed -i "s#DOMAIN_NAME#${DOMAIN_NAME}#g" /etc/nginx/conf.d/default.conf if [ $PANEL_TYPE == "0" ]; then sed -i "s#PORT#8080/functions/#g" /etc/nginx/conf.d/default.conf else sed -i "s#PORT#3000#g" /etc/nginx/conf.d/default.conf fi # restart nginx msg=$( #begin [[ $(systemctl is-active nginx.service) == "activating" ]] && systemctl reload nginx.service systemctl restart nginx 2>&1) status=$? updateProgress 50 "$msg" "$status" "nginx reconfig" } confignginx
为了让docker能覆盖安装,接下来脚本开头处逻辑的清空了配置,这里的重点问题是containerd与cni,与openfaasd的复杂关系:
Container Network Interface (CNI) 最早是由CoreOS发起的容器网络规范,是Kubernetes网络插件的基础。其基本思想为:Container Runtime在创建容器时,先创建好network namespace,然后调用CNI插件为这个netns配置网络,其后再启动容器内的进程。现已加入CNCF,成为CNCF主推的网络模型。CNI负责了在容器创建或删除期间的所有与网络相关的操作,它将创建所有规则以确保从容器进和出的网络连接正常,但它并不负责设置网络介质,例如创建网桥或分发路由以连接位于不同主机中的容器。
这个工作由openfaasd等完成。docker的这些组件->containerd+cni+ctr+runc,是由faasd来配置运行的。单独启动第一次安装完的containerd+cni+ctr+runc并不会启动cni和开启网卡(单独启动containerd提示cni conf not found没关系它依然会启动),需要openfaasd中的动作给后者带来cni和网卡配置。但这种结合很紧密,使得接下来容器的完全清理工作有难度。
对于容器的清除,用ctr tasks kill && ctr tasks delete && ctr container delete可以看到ps aux|grep manual看到主机空间的shim任务和/proc/id号/ns都被删掉了,但还是某些地方有残留。这是因为这二者很难分开,shim开启的task关联容器和/var/run/containerd无法清理,导致前者很难单独拔插/进行配置卸载,也难于在下一次覆盖安装时能从0全新开始。
而这其实是一个bug导致的,containerd cannot properly do "clean-up" with shim process during start up? #3971(https://github.com/containerd/containerd/issues/3971),直到1.4.0beta才被解决(https://github.com/containerd/containerd/pull/4100/commits/488d6194f2080709d9667e00ff244fbdc7ff95b2),但我测试了(cd /var/lib/faasd/ faasd up),只是效果好点,1.3.3是提示id exists不能重建container,1.40是提示/run/container下的files exists,同样没解决完全清理以全新覆盖安装containerd的需求,所以我脚本中提示了“containerd install+start progress(this may hang long and if you over install the script you may encount /run/containerd device busy error,for this case you need to reboot to fix after scripts finished”,这个基本如果你遇到了var/run删不掉错误,等安装程序跑完,重启即可。
因此我选择了1.40的containerd,它也同时解决了我开头提到的,gateway失效的问题。用的cni plugins还是0.8.5,本来想用那个cri-containerd-cni-1.4.0-linux-amd64.tar.gz,但里面的cni是0.7.1,与faasd要求的0.4.0不符。
对于cni的卸载和清除,则不属于ctr的能控制范畴,cni没有宿主机上的控制,除非将进程网络命名空间恢复到主机目录,或在在容器网络空间内运行IP命令来检查网络接口是否已正确设置,都挺麻烦,用上面删容器的ctr tasks kill && ctr tasks delete && ctr container delete三部曲删可以看到ifconfig中五个task对应的虚拟网卡也被干掉了,所以我也就没有再深入研究cni的卸载逻辑。
configdocker() { [[ $(systemctl is-active faasd-provider) == "activating" ]] && systemctl stop faasd-provider [[ $(systemctl is-active faasd) == "activating" ]] && systemctl stop faasd [[ $(systemctl is-active containerd) == "activating" ]] && ctr image remove docker.io/openfaas/basic-auth-plugin:0.18.18 docker.io/library/nats-streaming:0.11.2 docker.io/prom/prometheus:v2.14.0 docker.io/openfaas/gateway:0.18.18 docker.io/openfaas/queue-worker:0.11.2 && for i in basic-auth-plugin nats prometheus gateway queue-worker; do ctr tasks kill -s SIGKILL $i;ctr tasks delete $i;ctr container delete $i; done && systemctl stop containerd && sleep 10 ps -ef|grep containerd|awk '{print $2}'|xargs kill -9 rm -rf /var/run/containerd /run/containerd [[ ! -z "$(brctl show|grep openfaas0)" ]] && ifconfig openfaas0 down && brctl delbr openfaas0 rm -rf /etc/cni echo "===============================cniplugins installonly=================================" msg=$( #begin if [ ! -f "/tmp/cni-plugins-linux-amd64-v0.8.5.tar.gz" ]; then wget --no-check-certificate -qO- ${MIRROR_PATH}/docker/containernetworking/plugins/v0.8.5/cni-plugins-linux-amd64-v0.8.5.tar.gz > /tmp/cni-plugins-linux-amd64-v0.8.5.tar.gz fi mkdir -p /opt/cni/bin tar -xf /tmp/cni-plugins-linux-amd64-v0.8.5.tar.gz -C /opt/cni/bin /sbin/sysctl -w net.ipv4.conf.all.forwarding=1 2>&1) status=$? updateProgress 50 "$msg" "$status" "cniplugins installonly" echo "======containerd install+start progress(this may hang long and if you over install the script you may encount /run/containerd device busy error,for this case you need to reboot to fix after scripts finished)=====" msg=$( #begin # del original deb by docker.io rm -rf /usr/bin/containerd* /usr/bin/ctr # replace with new bins if [ ! -f "/tmp/containerd-1.4.0-linux-amd64.tar.gz" ]; then wget --no-check-certificate -qO- ${MIRROR_PATH}/docker/containerd/v1.4.0/containerd-1.4.0-linux-amd64.tar.gz > /tmp/containerd-1.4.0-linux-amd64.tar.gz fi tar -xf /tmp/containerd-1.4.0-linux-amd64.tar.gz -C ${INSTALL_DIR}/bin/ --strip-components=1 && ln -sf ${INSTALL_DIR}/bin/containerd* /usr/local/bin/ && ln -sf ${INSTALL_DIR}/bin/ctr /usr/local/bin/ctr rm -rf /etc/systemd/system/containerd.service cat << 'EOF' > /etc/systemd/system/containerd.service [Unit] Description=containerd container runtime Documentation=https://containerd.io After=network.target local-fs.target #After=network.target containerd.socket containerd.service #Requires=containerd.socket containerd.service [Service] ExecStartPre=-/sbin/modprobe overlay ExecStart=/usr/local/bin/containerd Type=notify Delegate=yes KillMode=process #changed to mixed to let systemctl stop containerd kill shims #KillMode=mixed Restart=always # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNPROC=infinity LimitCORE=infinity LimitNOFILE=1048576 # Comment TasksMax if your systemd version does not supports it. # Only systemd 226 and above support this version. TasksMax=infinity [Install] WantedBy=multi-user.target EOF systemctl daemon-reload && systemctl enable containerd systemctl start containerd --no-pager 2>&1) status=$? updateProgress 50 "$msg" "$status" "containerd install+start" } configdocker
到此,关于“nginx+docker怎么实现paas”的学习就结束了,希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习,快去试试吧!若想继续学习更多相关知识,请继续关注创新互联网站,小编会继续努力为大家带来更多实用的文章!
本文名称:nginx+docker怎么实现paas
URL地址:http://myzitong.com/article/pishej.html