쿠버네티스 프로젝트 5) MetalLB를 활용한 로드밸런서와 vote 어플리케이션 만들기

티스토리 뷰

쿠버네티스/프로젝트

쿠버네티스 프로젝트 5) MetalLB를 활용한 로드밸런서와 vote 어플리케이션 만들기

jws199 2025. 2. 1. 21:21

1. MetalLB: 오픈소스형 로드밸런서

1) 설치

[root@DESKTOP-ARLDN2P ~ (ubun01:nginx)]# helm repo add metallb https://metallb.github.io/metallb
[root@DESKTOP-ARLDN2P ~ (ubun01:nginx)]# helm repo list
NAME    URL
bitnami https://charts.bitnami.com/bitnami
metallb https://metallb.github.io/metallb
[root@DESKTOP-ARLDN2P ~ (ubun01:nginx)]# helm pull metallb/metallb --version 0.13.7
[root@DESKTOP-ARLDN2P ~ (ubun01:nginx)]# tar xvfz metallb-0.13.7.tgz

2) 설치

 [root@DESKTOP-ARLDN2P metallb (ubun01:metallb)]# helm install metallb -f values.yaml .
NAME: metallb
LAST DEPLOYED: Sat Feb  1 18:58:40 2025
NAMESPACE: metallb
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
MetalLB is now running in the cluster.

Now you can configure it via its CRs. Please refer to the metallb official docs
on how to use the CRs.
[root@DESKTOP-ARLDN2P metallb (ubun01:metallb)]# helm list
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART          APP VERSION
metallb metallb         1               2025-02-01 18:58:40.928374772 +0900 KST deployed        metallb-0.13.7 v0.13.7
[root@DESKTOP-ARLDN2P metallb (ubun01:metallb)]# k get svc,pod,deployments.apps,cm
NAME                              TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/metallb-webhook-service   ClusterIP   10.233.4.1   <none>        443/TCP   50s

NAME                                      READY   STATUS    RESTARTS   AGE
pod/metallb-controller-7495dc76b4-fkdk8   1/1     Running   0          50s
pod/metallb-speaker-dv7km                 1/1     Running   0          50s
pod/metallb-speaker-gk2sp                 1/1     Running   0          50s
pod/metallb-speaker-lkpr6                 1/1     Running   0          50s

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metallb-controller   1/1     1            1           50s

NAME                         DATA   AGE
configmap/kube-root-ca.crt   1      9m17s

2. vote application 설치

- 해당 어플리케이션은 간단한 투표 어플리케이션으로 해당 app을 로드밸런서와 연동할 것이다.

[root@DESKTOP-ARLDN2P metallb (ubun01:metallb)]# cd ..
[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# git clone https://github.com/dockersamples/example-voting-app.git
Cloning into 'example-voting-app'...
remote: Enumerating objects: 1179, done.
remote: Total 1179 (delta 0), reused 0 (delta 0), pack-reused 1179 (from 1)
Receiving objects: 100% (1179/1179), 1.21 MiB | 21.67 MiB/s, done.
Resolving deltas: 100% (448/448), done.
[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# cd example-voting-app/

- 어플리케이션이 설치될 네임스페이스 설정

[root@DESKTOP-ARLDN2P example-voting-app (ubun01:metallb)]# k create ns  vote
namespace/vote created
[root@DESKTOP-ARLDN2P example-voting-app (ubun01:metallb)]# k ns vote
Context "ubun01" modified.
Active namespace is "vote".
[root@DESKTOP-ARLDN2P example-voting-app (ubun01:vote)]# ls k8s-specifications/
db-deployment.yaml     redis-service.yaml      vote-deployment.yaml
db-service.yaml        result-deployment.yaml  vote-service.yaml
redis-deployment.yaml  result-service.yaml     worker-deployment.yaml
[root@DESKTOP-ARLDN2P example-voting-app (ubun01:vote)]#
[root@DESKTOP-ARLDN2P example-voting-app (ubun01:vote)]# k apply -f k8s-specifications/
deployment.apps/db created
service/db created
deployment.apps/redis created
service/redis created
deployment.apps/result created
service/result created
deployment.apps/vote created
service/vote created
deployment.apps/worker created

- 확인

[root@DESKTOP-ARLDN2P example-voting-app (ubun01:vote)]# k get po
NAME                      READY   STATUS    RESTARTS   AGE
db-74574d66dd-snfrs       1/1     Running   0          65s
redis-6c5fb9c4b7-w4ktq    1/1     Running   0          65s
result-5f99548f7c-8ppxk   1/1     Running   0          65s
vote-5d74dcd7c7-ndj9q     1/1     Running   0          65s
worker-6f5f6cdd56-zx9vb   1/1     Running   0          65s

- 설치된 서비스 변경(기존 서비스는 NodePort여서 LoadBalancer로 바뀌어야 한다)

vi k8s-specifications/vote-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: vote
  name: vote
spec:
  type: LoadBalancer # 수정
  ports:
  - name: "vote-service"
    port: 80 # 수정
    targetPort: 80
      #  nodePort: 31000 #수정
  selector:
    app: vote
k apply -f  k8s-specifications/vote-service.yaml

- 근데 문제가 생겼다. svc에 external-ip가 할당이 안된다.

[root@DESKTOP-ARLDN2P metallb (ubun01:vote)]# k get svc
NAME     TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
db       ClusterIP      10.233.60.40    <none>        5432/TCP         13m
redis    ClusterIP      10.233.3.203    <none>        6379/TCP         13m
result   NodePort       10.233.11.31    <none>        8081:31001/TCP   13m
vote     LoadBalancer   10.233.47.102   <pending>     80:31000/TCP     13m

- 찾아보니 address 관련한 커스텀 리소스가 metallb에서 필요하다

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-address-pool
  namespace: metallb
spec:
  addresses:
  - 172.17.29.70-172.17.29.85
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default-l2-advertisement
  namespace: metallb
  
  
kubectl apply -f metallb-config.yaml
kubectl delete pod -n metallb --all

- 재확인

[root@DESKTOP-ARLDN2P metallb (ubun01:metallb)]# k -n vote get svc
NAME     TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)          AGE
db       ClusterIP      10.233.60.40    <none>         5432/TCP         19m
redis    ClusterIP      10.233.3.203    <none>         6379/TCP         19m
result   NodePort       10.233.11.31    <none>         8081:31001/TCP   19m
vote     LoadBalancer   10.233.47.102   172.17.29.70   80:31000/TCP     19m

- 그리고 해당 external IP로 들어가면 application이 뜬다.

3. vote application 서버 운영

- 여러 pod가 설치되어 있다. 각 서버들 로그를 보다 편리하게 확인하기 위해 kubetail을 사용한다.

- kubetail 아래와 같이 인자를 주면 해당 인자를 가지고 있는 pod, 어플리케이션의 로그를 보여준다. -n 을 쓸 경우, namespace 기반으로 log 필터링이 진행된다.

[root@DESKTOP-ARLDN2P ~ (ubun01:vote)]# sudo apt -y install kubetail
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  kubetail
0 upgraded, 1 newly installed, 0 to remove and 82 not upgraded.
Need to get 8058 B of archives.
After this operation, 42.0 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu noble/universe amd64 kubetail all 1.6.18-1 [8058 B]
Fetched 8058 B in 1s (15.0 kB/s)
Selecting previously unselected package kubetail.
(Reading database ... 40804 files and directories currently installed.)
Preparing to unpack .../kubetail_1.6.18-1_all.deb ...
Unpacking kubetail (1.6.18-1) ...
Setting up kubetail (1.6.18-1) ...
Processing triggers for man-db (2.12.0-4build2) ...
[root@DESKTOP-ARLDN2P ~ (ubun01:vote)]# kubetail --version
1.6.18
[root@DESKTOP-ARLDN2P ~ (ubun01:vote)]# k ns metallb
Context "ubun01" modified.
Active namespace is "metallb".
[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# kubetail speaker
Will tail 3 logs...
metallb-speaker-f5stm
metallb-speaker-ls9bn
metallb-speaker-m2r5w
[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# kubetail -n vote
Will tail 7 logs...
db-74574d66dd-snfrs
redis-6c5fb9c4b7-w4ktq
result-5f99548f7c-8ppxk
vote-5d74dcd7c7-56g29
vote-5d74dcd7c7-jx4w6
vote-5d74dcd7c7-ndj9q
worker-6f5f6cdd56-zx9vb
[db-74574d66dd-snfrs] 2025-02-01 11:11:03.543 UTC [26] LOG:  checkpoint starting: time
[db-74574d66dd-snfrs] 2025-02-01 11:11:04.175 UTC [26] LOG:  checkpoint complete: wrote 7 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.612 s, sync=0.007 s, total=0.633 s; sync files=6, longest=0.003 s, average=0.002 s; distance=1 kB, estimate=1 kB
[vote-5d74dcd7c7-56g29] [2025-02-01 11:11:11,537] INFO in app: Received vote for b
[vote-5d74dcd7c7-56g29] [2025-02-01 11:11:11 +0000] [9] [INFO] Received vote for b
[vote-5d74dcd7c7-56g29] 10.233.109.128 - - [01/Feb/2025:11:11:11 +0000] "POST / HTTP/1.1" 200 1697 "http://172.17.29.70/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 Edg/132.0.0.0"
[vote-5d74dcd7c7-jx4w6] 10.233.109.128 - - [01/Feb/2025:11:11:11 +0000] "GET /static/stylesheets/style.css HTTP/1.1" 304 0 "http://172.17.29.70/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36 Edg/132.0.0.0"
[worker-6f5f6cdd56-zx9vb] Processing vote for 'b' by '21b29626aa9d71a'
[db-74574d66dd-snfrs] 2025-02-01 11:11:11.625 UTC [37] ERROR:  duplicate key value violates unique constraint "votes_id_key"
[db-74574d66dd-snfrs] 2025-02-01 11:11:11.625 UTC [37] DETAIL:  Key (id)=(21b29626aa9d71a) already exists.
[db-74574d66dd-snfrs] 2025-02-01 11:11:11.625 UTC [37] STATEMENT:  INSERT INTO votes (id, vote) VALUES ($1, $2)

- 또한 색깔별로 다양한 로그들을 보여주기에 기존 kubectl logs 보다 편리하다

4. 고가용성 테스트

1) k6를 이용한 부하 테스트

- k6 : CLI 환경, 자동화 스크립트 기반의 부하 테스트 제공

- k6 설치

sudo gpg -k
sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6

- 사용할 스크립트(k6를 위해선 스크립트 필요

k6-http.js
import http from 'k6/http';
import { check, group, sleep } from 'k6';

export let options = {
  stages: [
    { duration: '1m', target: 1000 }, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes.
    { duration: '2m', target: 1000 }, // stay at 100 users for 10 minutes
    { duration: '1m', target: 0 }, // ramp-down to 0 users
  ]
};

export default function () {
  http.get('172.17.29.70');
  sleep(1);
}


[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# vi k6-http.js
[root@DESKTOP-ARLDN2P ~ (ubun01:metallb)]# k6 run k6-http.js

- 총 4분에 걸친 부하테스트를 실행한다. ( 실 운영환경에선 통상 30분 정도 수행). 1분 준비, 2분 실행, 1분 종료 순서로, 1000개의 트랜잭션 부하를 생성

- http.get.IP http 부하를 실행할 IP를 LB 서비스의 External-IP로 지정

'쿠버네티스 > 프로젝트' 카테고리의 다른 글

[쿠버네티스] OpenEBS를 활용한 스토리지 관리 (0)	2025.02.19
쿠버네티스 프로젝트 6) Traefik (0)	2025.02.04
쿠버네티스 프로젝트 4) helm (0)	2025.02.01
쿠버네티스 프로젝트 3) 로컬 원격설정 (0)	2025.02.01
쿠버네티스 프로젝트 2) 클러스터 설치(kubespray) (1)	2025.01.29

공지사항

최근에 올라온 글

최근에 달린 댓글

Total

Today

Yesterday

링크

TAG more

« 2025/05 »
일	월	화	수	목	금	토
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

글 보관함

JW의 IT 블로그

티스토리 뷰

쿠버네티스 프로젝트 5) MetalLB를 활용한 로드밸런서와 vote 어플리케이션 만들기

'쿠버네티스 > 프로젝트' 카테고리의 다른 글

티스토리툴바