...
Visual Studio 설치 (옵션)
여기를 클릭하여 다운로드 및 설치한다.
...
Terraform을 사용하여 Azure Kubernetes Service로 Kubernetes 클러스터 만들기
소스 준비
| Code Block |
|---|
aks-demo# git clone https://github.com/zer0big/TerroformAksDeployDemo.git
Cloning into 'TerroformAksDeployDemo'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 15 (delta 2), reused 11 (delta 1), pack-reused 0
Unpacking objects: 100% (15/15), 9.43 KiB | 32.00 KiB/s, done.
aks-demo# cd TerroformAksDeployDemo/
aks-demo# code . |
|
Terraform 배포
az login terraform init terraform plan terraform apply -auto-approve |
| Code Block |
|---|
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ terraform init
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Installing hashicorp/azurerm v3.46.0...
- Installed hashicorp/azurerm v3.46.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ terraform plan
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_container_registry.zerobigaks-demo will be created
+ resource "azurerm_container_registry" "zerobigaks-demo" {
+ admin_enabled = false
+ admin_password = (sensitive value)
+ admin_username = (known after apply)
+ encryption = (known after apply)
+ export_policy_enabled = true
+ id = (known after apply)
+ location = "koreacentral"
+ login_server = (known after apply)
+ name = "tdgzeroacrdemo"
+ network_rule_bypass_option = "AzureServices"
+ network_rule_set = (known after apply)
+ public_network_access_enabled = true
+ resource_group_name = "RG-TDG-CMS-2023-AKSDemo"
+ retention_policy = (known after apply)
+ sku = "Standard"
+ trust_policy = (known after apply)
+ zone_redundancy_enabled = false
}
# azurerm_kubernetes_cluster.zerobigaks-demo will be created
+ resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
+ api_server_authorized_ip_ranges = (known after apply)
+ dns_prefix = "zerobigaksdemo"
+ fqdn = (known after apply)
+ http_application_routing_zone_name = (known after apply)
+ id = (known after apply)
+ image_cleaner_enabled = false
+ image_cleaner_interval_hours = 48
+ kube_admin_config = (sensitive value)
+ kube_admin_config_raw = (sensitive value)
+ kube_config = (sensitive value)
+ kube_config_raw = (sensitive value)
+ kubernetes_version = (known after apply)
+ location = "koreacentral"
+ name = "tdgzeroaksdemo"
+ node_resource_group = (known after apply)
+ oidc_issuer_url = (known after apply)
+ portal_fqdn = (known after apply)
+ private_cluster_enabled = false
+ private_cluster_public_fqdn_enabled = false
+ private_dns_zone_id = (known after apply)
+ private_fqdn = (known after apply)
+ public_network_access_enabled = true
+ resource_group_name = "RG-TDG-CMS-2023-AKSDemo"
+ role_based_access_control_enabled = true
+ run_command_enabled = true
+ sku_tier = "Free"
+ tags = {
+ "Environment" = "DEV"
}
+ workload_identity_enabled = false
+ default_node_pool {
+ kubelet_disk_type = (known after apply)
+ max_pods = (known after apply)
+ name = "default"
+ node_count = 2
+ node_labels = (known after apply)
+ orchestrator_version = (known after apply)
+ os_disk_size_gb = (known after apply)
+ os_disk_type = "Managed"
+ os_sku = (known after apply)
+ scale_down_mode = "Delete"
+ type = "VirtualMachineScaleSets"
+ ultra_ssd_enabled = false
+ vm_size = "Standard_D2_v2"
+ workload_runtime = (known after apply)
}
+ identity {
+ principal_id = (known after apply)
+ tenant_id = (known after apply)
+ type = "SystemAssigned"
}
}
# azurerm_resource_group.rg will be created
+ resource "azurerm_resource_group" "rg" {
+ id = (known after apply)
+ location = "koreacentral"
+ name = "RG-TDG-CMS-2023-AKSDemo"
}
# azurerm_role_assignment.enablePulling will be created
+ resource "azurerm_role_assignment" "enablePulling" {
+ id = (known after apply)
+ name = (known after apply)
+ principal_id = (known after apply)
+ principal_type = (known after apply)
+ role_definition_id = (known after apply)
+ role_definition_name = "AcrPull"
+ scope = (known after apply)
+ skip_service_principal_aad_check = true
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ aks_fqdn = (known after apply)
+ aks_id = (known after apply)
+ aks_node_rg = (known after apply)
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply"
now.
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ terraform apply -auto-approve
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_container_registry.zerobigaks-demo will be created
+ resource "azurerm_container_registry" "zerobigaks-demo" {
+ admin_enabled = false
+ admin_password = (sensitive value)
+ admin_username = (known after apply)
+ encryption = (known after apply)
+ export_policy_enabled = true
+ id = (known after apply)
+ location = "koreacentral"
+ login_server = (known after apply)
+ name = "tdgzeroacrdemo"
+ network_rule_bypass_option = "AzureServices"
+ network_rule_set = (known after apply)
+ public_network_access_enabled = true
+ resource_group_name = "RG-TDG-CMS-2023-AKSDemo"
+ retention_policy = (known after apply)
+ sku = "Standard"
+ trust_policy = (known after apply)
+ zone_redundancy_enabled = false
}
# azurerm_kubernetes_cluster.zerobigaks-demo will be created
+ resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
+ api_server_authorized_ip_ranges = (known after apply)
+ dns_prefix = "zerobigaksdemo"
+ fqdn = (known after apply)
+ http_application_routing_zone_name = (known after apply)
+ id = (known after apply)
+ image_cleaner_enabled = false
+ image_cleaner_interval_hours = 48
+ kube_admin_config = (sensitive value)
+ kube_admin_config_raw = (sensitive value)
+ kube_config = (sensitive value)
+ kube_config_raw = (sensitive value)
+ kubernetes_version = (known after apply)
+ location = "koreacentral"
+ name = "tdgzeroaksdemo"
+ node_resource_group = (known after apply)
+ oidc_issuer_url = (known after apply)
+ portal_fqdn = (known after apply)
+ private_cluster_enabled = false
+ private_cluster_public_fqdn_enabled = false
+ private_dns_zone_id = (known after apply)
+ private_fqdn = (known after apply)
+ public_network_access_enabled = true
+ resource_group_name = "RG-TDG-CMS-2023-AKSDemo"
+ role_based_access_control_enabled = true
+ run_command_enabled = true
+ sku_tier = "Free"
+ tags = {
+ "Environment" = "DEV"
}
+ workload_identity_enabled = false
+ default_node_pool {
+ kubelet_disk_type = (known after apply)
+ max_pods = (known after apply)
+ name = "default"
+ node_count = 2
+ node_labels = (known after apply)
+ orchestrator_version = (known after apply)
+ os_disk_size_gb = (known after apply)
+ os_disk_type = "Managed"
+ os_sku = (known after apply)
+ scale_down_mode = "Delete"
+ type = "VirtualMachineScaleSets"
+ ultra_ssd_enabled = false
+ vm_size = "Standard_D2_v2"
+ workload_runtime = (known after apply)
}
+ identity {
+ principal_id = (known after apply)
+ tenant_id = (known after apply)
+ type = "SystemAssigned"
}
}
# azurerm_resource_group.rg will be created
+ resource "azurerm_resource_group" "rg" {
+ id = (known after apply)
+ location = "koreacentral"
+ name = "RG-TDG-CMS-2023-AKSDemo"
}
# azurerm_role_assignment.enablePulling will be created
+ resource "azurerm_role_assignment" "enablePulling" {
+ id = (known after apply)
+ name = (known after apply)
+ principal_id = (known after apply)
+ principal_type = (known after apply)
+ role_definition_id = (known after apply)
+ role_definition_name = "AcrPull"
+ scope = (known after apply)
+ skip_service_principal_aad_check = true
}
Plan: 4 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ aks_fqdn = (known after apply)
+ aks_id = (known after apply)
+ aks_node_rg = (known after apply)
azurerm_resource_group.rg: Creating...
azurerm_resource_group.rg: Creation complete after 1s [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo]
azurerm_container_registry.zerobigaks-demo: Creating...
azurerm_kubernetes_cluster.zerobigaks-demo: Creating...
azurerm_container_registry.zerobigaks-demo: Still creating... [10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [10s elapsed]
azurerm_container_registry.zerobigaks-demo: Creation complete after 17s [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Creation complete after 3m45s [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo]
azurerm_role_assignment.enablePulling: Creating...
azurerm_role_assignment.enablePulling: Still creating... [10s elapsed]
azurerm_role_assignment.enablePulling: Still creating... [20s elapsed]
azurerm_role_assignment.enablePulling: Creation complete after 26s [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo/providers/Microsoft.Authorization/roleAssignments/41ac72ed-cc01-d35e-a12c-23c90fbebfe6]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
aks_fqdn = "zerobigaksdemo-9fctjqgz.hcp.koreacentral.azmk8s.io"
aks_id = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo"
aks_node_rg = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral" |
|
Terraform 배포 결과 확인
Azure Portal로 이동하여 배포 결과를 확인한다.
...
ACR에 대한 AKS 관리 ID Role 부여 현황을 확인한다.
...
AKS 클러스터 검증
Application 배포 및 검증
ACR 이미지 등록
별도 터미널 창을 열어 이미지 태그 및 푸시
| Code Block |
|---|
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
새로운 크로스 플랫폼 PowerShell 사용 https://aka.ms/pscore6
PS C:\Users\zerobig> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
PS C:\Users\zerobig> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
appmod-contapp latest 8629e2fcf972 4 days ago 121MB
zeroacrdemo.azurecr.io/ap latest 8629e2fcf972 4 days ago 121MB
zeroacrdemo.azurecr.io/appmod-contapp latest 8629e2fcf972 4 days ago 121MB
zeroacrtf.azurecr.io/appmod-contapp latest 8629e2fcf972 4 days ago 121MB
PS C:\Users\zerobig> docker tag appmod-contapp osd2023zeroacrdemo.azurecr.io/appmod-contapp:latest
PS C:\Users\zerobig> az acr login -n osd2023zeroacrdemo
Login Succeeded
PS C:\Users\zerobig> docker push osd2023zeroacrdemo.azurecr.io/appmod-contapp:latest
The push refers to repository [osd2023zeroacrdemo.azurecr.io/appmod-contapp]
7374f8b571a9: Pushed
ceaf9e1ebef5: Pushed
9b9b7f3d56a0: Pushed
f1b5933fe4b5: Pushed
latest: digest: sha256:ddf9994cb1630255c2fcec8b60617811ca751e59aaf7bcc99677028cd43f78c1 size: 1159
PS C:\Users\zerobig> |
|
YAML Manifest 구성
| Code Block |
|---|
apiVersion: apps/v1
kind: Deployment
metadata:
name: zeroaksdemo
spec:
replicas: 2
selector:
matchLabels:
app: zero-aks-app
template:
metadata:
labels:
app: zero-aks-app
spec:
containers:
- name: zero-aks-demo
image: zeroacrtf.azurecr.io/web:latest
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: zero-aks-app
spec:
ports:
- name: http-port
port: 80
targetPort: 8080
selector:
app: zero-aks-app
type: LoadBalancer |
|
YAML Manifest 배포 및 결과 검증
| Code Block |
|---|
zerobig@ZEROBIG-NT800:/mnt/d/2023_Azure/HandsOn/Terraform$ kubectl apply -f contapp-deploy-svc.yaml
deployment.apps/zeroaksdemo created
service/zero-aks-app created
zerobig@ZEROBIG-NT800:/mnt/d/2023_Azure/HandsOn/Terraform$ kubectl get pod --watch
NAME READY STATUS RESTARTS AGE
zeroaksdemo-5874694c78-2k6zk 1/1 Running 0 7s
zeroaksdemo-5874694c78-4xrbt 1/1 Running 0 7s
^Czerobig@ZEROBIG-NT800:/mnt/d/2023_Azure/HandsOn/Terraform$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 17m
zero-aks-app LoadBalancer 10.0.104.234 20.214.218.34 80:30193/TCP 3m2s
zerobig@ZEROBIG-NT800:/mnt/d/2023_Azure/HandsOn/Terraform$ |
|