k8s安裝ingress nginx

ingress-nginx最新版已經不包含default-http-backend.
因此安裝0.20.0

#安裝ingress-nginx
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.20.0/deploy/mandatory.yaml
sed -i 's/serviceAccountName: nginx-ingress-serviceaccount/hostNetwork: true\n      serviceAccountName: nginx-ingress-serviceaccount/g' mandatory.yaml
kubectl apply -f mandatory.yaml
rm -f mandatory.yaml*
kubectl get pod --all-namespaces

#修改ingress-nginx
NODE_COUNT=$(kubectl get nodes | grep -v master | grep -v STATUS | wc -l)
echo $NODE_COUNT
if [ $NODE_COUNT -gt 1 ] ; then
kubectl -n ingress-nginx patch deployment default-http-backend --patch $(echo "{\"spec\":{\"replicas\":$NODE_COUNT}}")
kubectl -n ingress-nginx patch deployment nginx-ingress-controller --patch $(echo "{\"spec\":{\"replicas\":$NODE_COUNT}}")
fi
kubectl get pods -n ingress-nginx -o wide

#更換自製 http-backend image
DOMAIN=ssl.cbe.tw
kubectl -n ingress-nginx patch deployment default-http-backend --patch "{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"default-http-backend\",\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"200Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"200Mi\"}},\"image\":\"slanla/apache-defaultbackend\",\"ports\":[{\"containerPort\":8080,\"protocol\":\"TCP\"}],\"env\":[{\"name\":\"LETSENCRYPT_PROXYPASS_URL\",\"value\":\"http://$DOMAIN/.well-known/acme-challenge/ connectiontimeout=15 timeout=30\"},{\"name\":\"LETSENCRYPT_PROXYPASSREVERSE_URL\",\"value\":\"http://$DOMAIN/.well-known/acme-challenge/\"}],\"livenessProbe\":{\"httpGet\":{\"path\":\"/healthz\",\"port\":8080,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":30,\"timeoutSeconds\":5,\"periodSeconds\":10,\"successThreshold\":1,\"failureThreshold\":3}}]}}}}"
kubectl get pods -n ingress-nginx -o wide

刪除所有k8s上面的pod

刪除所有k8s上面的pod

kubectl get pods --all-namespaces -o wide | awk '{print $1 " " $2}' | while read AA BB; do kubectl delete pod --grace-period=0 --force -n $AA $BB; done

刪除所有k8s上面所有非Running的pod

kubectl get pods --all-namespaces -o wide | grep -v Running | awk '{print $1 " " $2}' | while read AA BB; do kubectl delete pod --grace-period=0 --force -n $AA $BB; done

使用helm部署nfs-client

之前都用yaml部署nfs-client.
但缺點是每次kubernets跟nfs-client版本更換之後,
就可能會發生部署有問題,像是權限…等

剛剛在kubernetes 1.12上面用之前1.10所用的nfs-client之yaml檔案.
結果又出問題了.
後來發現helm有提供nfs-client部署方式.
二話不說,立刻改用helm部署.
語法如下:

helm install stable/nfs-client-provisioner \
  --name nfs-client \
  --set nfs.server=xxx.xxx.xxx.xxx \
  --set nfs.path=/path \
  --set storageClass.name=managed-nfs-storage

mongDB帳號設定

參考下列指令:

# 環境準備
docker rm -f mongodb_mongo_1
rm -r -f $PWD/db
mkdir -p $PWD/db

# 建立mongoDB
docker run \
  -d \
  --name mongodb_mongo_1 \
  -p 27017:27017 \
  -v $PWD/db:/data/db \
  mongo
sleep 5

# 建立超級管理者
docker exec -it mongodb_mongo_1 bash
mongo admin
db.createUser({ user: "admin" , pwd: "admin1234", roles: ["userAdminAnyDatabase", "dbAdminAnyDatabase", "readWriteAnyDatabase"]})
quit()
# 重新登入
mongo admin
db.auth("admin","admin1234")
use octblog
db.createUser({
    user: "gevin",
    pwd: "gevin1234",
    roles: [ { role: "readWrite", db: "octblog" },
             { role: "readWrite", db: "octblog-log" } ]
})
quit()

mongo octblog
db.auth("gevin","gevin1234")
use octblog
db.collection.insert({'test':1234});
quit()
exit

# 重啟mongoDB
docker rm -f mongodb_mongo_1
docker run \
  -d \
  --name mongodb_mongo_1 \
  -p 27017:27017 \
  -v $PWD/db:/data/db \
  mongo mongod --auth

重新產生k8s join指令

因為新版的k8s的token已經有時間效期,
所以在安裝完後,隔一陣子想要加入新的節點就會出現舊的join token無法使用.
這時候可以用kubeadm token generate重新產生新的token,並用下列語法印出join指令.

kubeadm token create `kubeadm token generate` --print-join-command --ttl=0

以helm在k8s上部署mariaDB

環境變數:

PASSWORD="1234567890"
NAME="db"
SERVICE="${NAME}-mariadb"
NAMESPACE="default"

儲存裝置為nfs

helm install \
  --namespace ${NAMESPACE} \
  --name ${NAME} stable/mariadb \
  --set slave.replicas=3 \
  --set rootUser.password=replicator \
  --set replication.user="${PASSWORD}" \
  --set replication.password="${PASSWORD}" \
  --set master.persistence.storageClass=managed-nfs-storage \
  --set slave.persistence.storageClass=managed-nfs-storage

參數參考

允許root@%登入(注意:密碼不能隨意變動)

kubectl exec -it -n ${NAMESPACE} ${SERVICE}-master-0 -- mysql -uroot -p${PASSWORD} -e "grant all privileges on *.* to 'root' @'%' identified by '${PASSWORD}'; FLUSH PRIVILEGES;"

變更密碼正確方式:

NEW_PASSWORD="0987654321"
PASSWORD=$(kubectl get secret -n ${NAMESPACE} ${SERVICE} -o jsonpath="{.data.mariadb-root-password}" | base64 --decode)
echo "Password: ${PASSWORD}"
helm upgrade ${NAME} stable/mariadb --set rootUser.password=${NEW_PASSWORD}

MongoDB Replica Set高可用性的建置

  1. 建立三個MongoDB
echo "[info] 建立測試環境";
docker network create mongo-cluster >/dev/null 2>&1
docker rm -f mongo1 mongo2 mongo3 >/dev/null 2>&1
for i in 1 2 3
do
  echo "       mongo${i} 建立"
  docker run \
    -d \
    -p 3000$i:27017 \
    --name mongo$i \
    --net mongo-cluster \
    mongo mongod --replSet my-mongo-set >/dev/null 2>&1
done
  1. 接著設定三者關係,並強制設定electionTimeoutMillis為500ms.
echo "       設定mongo cluster"
docker exec -it mongo3 mongo --eval "config={\"_id\":\"my-mongo-set\",\"members\":[{\"_id\":0,\"host\":\"mongo1:27017\"},{\"_id\":1,\"host\":\"mongo2:27017\"},{\"_id\":2,\"host\":\"mongo3:27017\"}],\"settings\": { \"electionTimeoutMillis\": 500 }}; rs.initiate(config)"
  1. PHP測試
#https://github.com/mongodb/mongo-php-library
require_once __DIR__ . "/vendor/autoload.php";

try{
  $client = new MongoDB\Client("mongodb://mongo1:27017,mongo2:27017,mongo3:27017",[],[
    'typeMap' => [
      'array' => 'array',
      'document' => 'array',
      'root' => 'array',
    ],
  ]);
  $db = $client->selectDatabase('test');
  $cursor = $db->command(['isMaster' => 1]);
  echo "[info] 成功連線至: {$cursor->toArray()[0]['primary']}\n";

} catch(Exception $e) {
  echo "[error] 無法線連\n";
  exit;
}

{  //若Collection:testA不存在則建立.
  if((iterator_count($db->listCollections([
    'filter'=>[
      'name'=>'testA'
    ]
  ])))==0) //當數量為0時,建立testA
    $db->createCollection('testA');
}

//連線至Collection:testA
$collection=$db->selectCollection('testA');

//若沒有任何資料,則建立100則隨機資料
if($collection->count()==0){
  for($i=0;$i<100;$i++){
    $insertOneResult = $collection->insertOne([
      'idx' => $i,
      'value' => "{$i}, rand: ".rand(0,9999),
    ]);
  }
}

//取得最後三筆資料
foreach($collection->find([],[
  'limit' => 3,
  'projection' => ['_id'=>0],
  'sort' => [
    'idx'=>-1
  ]
]) as $rs){
  echo "\tidx:{$rs['idx']}\tvalue:{$rs['value']}\n";
}

刪除無作用的docker image

在使用docker build建立image的過程中,往往會產生很多個none的docker images

因此可以透過下列指令刪除

docker rmi $(docker images | grep none | awk '{print $3}')