AWS ALB 삭제 후 Nginx Ingress(Baremetal) 구축 (요금 문제) #42
Assignees
Labels
Comments
Closed
기본 개념Ingress에 대해서Ingress는 외부에서 k8s cluster 내부로 들어오는 네트워크 요청 즉, <Ingress 트래픽>을 어떻게 처리할지 정의한다.
어떻게 실제 구현하는가
Ingress (Object) 와 Ingress Controller 차이k8s 에서 Ingress 를 사용하기 위해서는 두 가지가 필요하다.
Ingress 는 단지 Ingress 규칙을 정의하는 선언적인 오브젝트일 뿐, 외부 요청을 받아들이는 실제 서버가 아니기 때문이다. Ingress 는 Ingress Controller 라고 하는 특수한 서버 컨테이너에 적용되어야 Ingress 에 적용된 규칙이 활성화된다.
Ingress-nginx
중간정리: 그렇다면 나는 어떻게 해야 하는가?
MetalLB란?출처 Nodeport
NodePort는 효율성이 떨어진다.
만약 노드가 죽는다면, IP가 달라지는 문제점이 생긴다.
Load Balancer on Cloud
가장 간단하게 동작한다
MetalLB만약 클라우드가 아닌 환경에서 LoadBalancer 타입을 배포하면 어떻게 될까?
어떻게 할 수 있을까? MetalLB를 통해
핵심 컴퍼넌트 Speaker는 모든 노드에 실행되도록 daemonset 사용 - 특히 **host network**를 이용한다. - L2 모드는 **리더 Speaker Pod**이 존재한다. (리더 Specker Pod의 Node의 Externel IP가 대표 IP)
다양한 위치에 배포되는 MetalLB
중간정리: MetalLB의 필요성현재상황
필요성
Todo: Ingresss-nginx + HostNetwork 조합은 불가능한가?
|
AWS Load Balancer 없이 EKS Node(EC2)의 Public IP로 Nginx Ingress에 집적 붙을 수 는 없나?키워드
How do I make ingress-nginx work without built-in LoadBalancer support for Kubernetes?I would need to expose services so that outside world can reach it - which can be done with I can just expose ingress-nginx as NodePort and point DNS record (AAAA) to it? "mysebsite.com" with AAAA record of "123.456.789", the nginx-ingress exposed to "123.456.789" will receive the request
EKS 각 Node의 외부접근(특정 포트)을 허용한다각각의 $ kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-71-165.ap-northeast-2.compute.internal Ready <none> 23h v1.27.4-eks-8ccc7ba 10.0.71.165 13.125.122.142 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-77-252.ap-northeast-2.compute.internal Ready <none> 2d6h v1.27.4-eks-8ccc7ba 10.0.77.252 43.201.85.206 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-93-81.ap-northeast-2.compute.internal Ready <none> 2d6h v1.27.4-eks-8ccc7ba 10.0.93.81 43.201.150.44 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-94-57.ap-northeast-2.compute.internal Ready <none> 2d6h v1.27.4-eks-8ccc7ba 10.0.94.57 52.79.49.224 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
EKS Node Group SG 변경module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = "ticketing-main-eks"
...
+ node_security_group_additional_rules = {
+ ingress_http = {
+ description = "open http"
+ protocol = "-1"
+ from_port = 80
+ to_port = 80
+ type = "ingress"
+ cidr_blocks = ["0.0.0.0/0"]
+ }
+ }
}
Terraform AWS VPC 설정 변경Network Access Control Lists (ACL or NACL)에 관련 설명이 존재한다. module "vpc" {
source = "terraform-aws-modules/vpc/aws"
name = "main"
cidr = "10.0.0.0/16"
azs = ["ap-northeast-2a", "ap-northeast-2b"]
private_subnets = ["10.0.0.0/19", "10.0.32.0/19"]
public_subnets = ["10.0.64.0/19", "10.0.96.0/19"]
map_public_ip_on_launch = true
+ manage_default_network_acl = true
+ public_dedicated_network_acl = true
+ public_inbound_acl_rules = [
+ {
+ rule_number = 1
+ rule_action = "allow"
+ from_port = 80
+ to_port = 80
+ protocol = "tcp"
+ cidr_block = "0.0.0.0/0"
+ },
+ ]
}그 외, 고려해볼 VPC 옵션 기존
규칙 우선순위
테스트 Nodeport:30000 으로 연결 가능 테스트apiVersion: v1
kind: Service
metadata:
name: test
spec:
type: NodePort # 서비스 타입
ports:
- port: 8080
targetPort: 8080
nodePort: 30001
protocol: TCP
name: http
externalIPs: # 만약 이 부분이 없어진다면 접속에 장애가 생긴다
- 13.125.122.142
- 43.201.85.206
selector:
app.kubernetes.io/instance: prod
app.kubernetes.io/name: ticketing-applicationhostNetwork: true를 이용한 80 포트 접속 테스트 |
Merged
Merged
Nginx Ingress Baremetal 직접 따라해보기참고 배경: Backend ServiceapiVersion: v1
kind: Service
metadata:
name: {{ include "ticketing-chart.fullname" . }}
spec:
type: ClusterIP
ports:
- port: {{ .Values.backend.port }}
targetPort: {{ .Values.backend.port }}
selector:
{{- include "ticketing-chart.selectorLabels" . | nindent 4 }}$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 172.20.0.1 <none> 443/TCP 30d
prod-ticketing-application ClusterIP 172.20.110.40 <none> 8080/TCP 15s
$ kubectl port-forward svc/prod-ticketing-application 3001:8080
Forwarding from 127.0.0.1:3001 -> 8080
Forwarding from [::1]:3001 -> 8080
$ curl 127.0.0.1:3001
OK
$ curl 127.0.0.1:3001/bookamrks
{"message":"JWT Autorization failed."}따라해보기$ wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/baremetal/deploy.yaml
$ kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
$ kubectl get all -n=ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-pxrtx 0/1 Completed 0 9s
pod/ingress-nginx-admission-patch-ff6xm 0/1 Completed 0 8s
pod/ingress-nginx-controller-79bc9f5df8-7g9tv 0/1 Running 0 9s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 172.20.109.252 <none> 80:31475/TCP,443:31396/TCP 9s
service/ingress-nginx-controller-admission ClusterIP 172.20.144.89 <none> 443/TCP 9s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 0/1 1 0 10s
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-79bc9f5df8 1 1 0 10s
NAME COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create 1/1 4s 10s
job.batch/ingress-nginx-admission-patch 1/1 3s 9sapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: prod-ticketing-application
port:
number: 8080
ingressClassName: nginx$ kubectl apply -f ingress.yaml
ingress.networking.k8s.io/ingress-myservicea created
$ kubectl describe ingress ingress-myservicea
Name: ingress-myservicea
Labels: <none>
Namespace: default
Address:
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
*
/ prod-ticketing-application:8080 (10.0.87.162:8080)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 5s nginx-ingress-controller Scheduled for sync
$ kubectl get svc -n=ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 172.20.109.252 <none> 80:31475/TCP,443:31396/TCP 2m10s
ingress-nginx-controller-admission ClusterIP 172.20.144.89 <none> 443/TCP 2m10s
$ kubectl get svc -n=ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 172.20.109.252 <none> 80:31475/TCP,443:31396/TCP 2m10s
ingress-nginx-controller-admission ClusterIP 172.20.144.89 <none> 443/TCP 2m10s
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-71-165.ap-northeast-2.compute.internal Ready <none> 45h v1.27.4-eks-8ccc7ba 10.0.71.165 13.125.122.142 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-77-252.ap-northeast-2.compute.internal Ready <none> 3d4h v1.27.4-eks-8ccc7ba 10.0.77.252 43.201.85.206 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-93-81.ap-northeast-2.compute.internal Ready <none> 3d4h v1.27.4-eks-8ccc7ba 10.0.93.81 43.201.150.44 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
ip-10-0-94-57.ap-northeast-2.compute.internal Ready <none> 3d4h v1.27.4-eks-8ccc7ba 10.0.94.57 52.79.49.224 Amazon Linux 2 5.10.186-179.751.amzn2.x86_64 containerd://1.6.19
$ curl 13.125.122.142:31475
OK
$ curl 43.201.85.206:31475
OK기타
HostNetwork True 구성 따라하기Template 파일 생성 $ helm template ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --set controller.hostNetwork=true,controller.service.type="",controller.kind=DaemonSet > nginx.yamlapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-myservicea
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: development-ticketing-application
port:
number: 8080
ingressClassName: nginxTest$ kubectl describe ingress 05:15:21 PM
Name: development-ticketing-application
Labels: argocd.argoproj.io/instance=development
Namespace: default
Address: 172.20.209.236
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
*
/ development-ticketing-application:8080 (10.0.80.146:8080)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 29m (x2 over 30m) nginx-ingress-controller Scheduled for sync
Normal Sync 29m (x2 over 30m) nginx-ingress-controller Scheduled for sync
Normal Sync 29m (x2 over 30m) nginx-ingress-controller Scheduled for sync
Normal Sync 29m (x2 over 30m) nginx-ingress-controller Scheduled for sync
❯ curl 43.201.85.206
OKTodo
|
junha-ahn
changed the title
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
#37 중 중요한 이슈 발견
ALB는 사용당 (커넥션당) 요금이 발생한다.
부하테스트 중 심각한 요금 문제가 발생할 가능성이 높음으로, ALB를 제거한다.
기존구조
IG => ALB(Public-IP) => EKS Backend Pod => DB향후구조
IG => EKS NginX Ingress Service (Public-IP) => EKS Backend Pod => DBTo do
Test Checklist
curl ...:80/bookmarksreturn[]Result
The text was updated successfully, but these errors were encountered: