Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Table of Contents
minLevel1
maxLevel6
outlinefalse
typelist
printablefalse

Azure SA 업무환경 설정

Azure VM 생성 - Windows 11

구독, 리소스 그룹을 선택 및 입력한다.

...

Code Block
# Install HashiCorp's Debian package repository
sudo apt-get update && sudo apt-get install -y gnupg software-properties-commocommon

# Install the HashiCorp GPG key
wget -O- https://apt.releases.hashicorp.com/gpg | \
    gpg --dearmor | \
    sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg

# Verify the key's fingerprint.
gpg --no-default-keyring \
    --keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
    --fingerprint

# Add the official HashiCorp repository to your system
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
    https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
    sudo tee /etc/apt/sources.list.d/hashicorp.list

# Download the package information and Install Terraform from the new repository.
sudo apt update
sudo apt-get install terraform

# Verify the installation
terraform -v

# Enable tab completion
terraform -install-autocomplete
source ~/.bashrc

...

먼저 여기에서 최신 버전을 다운로드 받고 기본으로 설치를 진행한다. ← 32bit

64bit는 여기

For Linux(WSL)

다음 명령을 수행하여 설치를 진행하고 결과를 검증한다.

...

Visual Studio 설치 (옵션)

여기를 클릭하여 다운로드 및 설치한다.

...

Terraform을 사용하여 Azure Kubernetes Service로 Kubernetes 클러스터 만들기

소스 준비

git clone https://github.com/zer0big/

TerroformAksDeployDemo

TDG-TerraformAKSDemo.git

cd TerroformAksDeployDemo/

code .

Code Block
aks
tdg-zerobig-
demo#
wsls# git clone https://github.com/zer0big/
TerroformAksDeployDemo
TDG-TerraformAKSDemo.git
Cloning into '
TerroformAksDeployDemo
TDG-TerraformAKSDemo'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 15 (delta 
2
0), reused 
11
15 (delta 
1
0), pack-reused 0
Unpacking
Receiving objects: 100% (15/15), 
9
18.
43
39 KiB | 
32
570.00 KiB/s, done.
aks
tdg-zerobig-
demo#
wsls# cd 
TerroformAksDeployDemo
TDG-TerraformAKSDemo/
aks
tdg-zerobig-
demo# code
wsls# code .

Terraform 배포

az login

terraform init

terraform plan

terraform apply -auto-approve

Code Block
tdg-zerobig-wsls# terraform plan
azurerm_resource_group.rg: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo]
azurerm_container_registry.zerobigaks-demo: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo]
azurerm_kubernetes_cluster.zerobigaks-demo: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo]
azurerm_role_assignment.enablePulling: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo/providers/Microsoft.Authorization/roleAssignments/41ac72ed-cc01-d35e-a12c-23c90fbebfe6]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply"
which may have affected this plan:

  # azurerm_container_registry.zerobigaks-demo has been deleted
  - resource "azurerm_container_registry" "zerobigaks-demo" {
      - id                            = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo" -> null
        name                          = "tdgzeroacrdemo"
        # (16 unchanged attributes hidden)
    }

  # azurerm_kubernetes_cluster.zerobigaks-demo has been deleted
  - resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
      - fqdn                                = "zerobigaksdemo-9fctjqgz.hcp.koreacentral.azmk8s.io" -> null
      - id                                  = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo" -> null
        name                                = "tdgzeroaksdemo"
      - node_resource_group                 = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral"
 -> null
        tags                                = {
            "Environment" = "DEV"
        }
        # (21 unchanged attributes hidden)

      - kubelet_identity {
          - object_id                 = "02a2ad14-8c56-420c-ba60-e95b16680a98" -> null
            # (2 unchanged attributes hidden)
        }

        # (3 unchanged blocks hidden)
    }

  # azurerm_resource_group.rg has been deleted
  - resource "azurerm_resource_group" "rg" {
        id       = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo"
      - location = "koreacentral" -> null
      - name     = "RG-TDG-CMS-2023-AKSDemo" -> null
    }


Unless you have made equivalent changes to your configuration, or ignored the relevant attributes
using ignore_changes, the following plan may include actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_container_registry.zerobigaks-demo will be created
  + resource "azurerm_container_registry" "zerobigaks-demo" {
      + admin_enabled                 = false
      + admin_password                = (sensitive value)
      + admin_username                = (known after apply)
      + encryption                    = (known after apply)
      + export_policy_enabled         = true
      + id                            = (known after apply)
      + location                      = "koreacentral"
      + login_server                  = (known after apply)
      + name                          = "tdgzeroacrdemo"
      + network_rule_bypass_option    = "AzureServices"
      + network_rule_set              = (known after apply)
      + public_network_access_enabled = true
      + resource_group_name           = "RG-TDG-CMS-2023-AKSDemo"
      + retention_policy              = (known after apply)
      + sku                           = "Standard"
      + trust_policy                  = (known after apply)
      + zone_redundancy_enabled       = false
    }

  # azurerm_kubernetes_cluster.zerobigaks-demo will be created
  + resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
      + api_server_authorized_ip_ranges     = (known after apply)
      + dns_prefix                          = "zerobigaksdemo"
      + fqdn                                = (known after apply)
      + http_application_routing_zone_name  = (known after apply)
      + id                                  = (known after apply)
      + image_cleaner_enabled               = false
      + image_cleaner_interval_hours        = 48
      + kube_admin_config                   = (sensitive value)
      + kube_admin_config_raw               = (sensitive value)
      + kube_config                         = (sensitive value)
      + kube_config_raw                     = (sensitive value)
      + kubernetes_version                  = (known after apply)
      + location                            = "koreacentral"
      + name                                = "tdgzeroaksdemo"
      + node_resource_group                 = (known after apply)
      + oidc_issuer_url                     = (known after apply)
      + portal_fqdn                         = (known after apply)
      + private_cluster_enabled             = false
      + private_cluster_public_fqdn_enabled = false
      + private_dns_zone_id                 = (known after apply)
      + private_fqdn                        = (known after apply)
      + public_network_access_enabled       = true
      + resource_group_name                 = "RG-TDG-CMS-2023-AKSDemo"
      + role_based_access_control_enabled   = true
      + run_command_enabled                 = true
      + sku_tier                            = "Free"
      + tags                                = {
          + "Environment" = "DEV"
        }
      + workload_identity_enabled           = false

      + default_node_pool {
          + kubelet_disk_type    = (known after apply)
          + max_pods             = (known after apply)
          + name                 = "default"
          + node_count           = 2
          + node_labels          = (known after apply)
          + orchestrator_version = (known after apply)
          + os_disk_size_gb      = (known after apply)
          + os_disk_type         = "Managed"
          + os_sku               = (known after apply)
          + scale_down_mode      = "Delete"
          + type                 = "VirtualMachineScaleSets"
          + ultra_ssd_enabled    = false
          + vm_size              = "Standard_D2_v2"
          + workload_runtime     = (known after apply)
        }

      + identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = "SystemAssigned"
        }
    }

  # azurerm_resource_group.rg will be created
  + resource "azurerm_resource_group" "rg" {
      + id       = (known after apply)
      + location = "koreacentral"
      + name     = "RG-TDG-CMS-2023-AKSDemo"
    }

  # azurerm_role_assignment.enablePulling will be created
  + resource "azurerm_role_assignment" "enablePulling" {
      + id                               = (known after apply)
      + name                             = (known after apply)
      + principal_id                     = (known after apply)
      + principal_type                   = (known after apply)
      + role_definition_id               = (known after apply)
      + role_definition_name             = "AcrPull"
      + scope                            = (known after apply)
      + skip_service_principal_aad_check = true
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  ~ aks_fqdn    = "zerobigaksdemo-9fctjqgz.hcp.koreacentral.azmk8s.io" -> (known after apply)
  ~ aks_id      = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo" -> (known after apply)
  ~ aks_node_rg = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral" -> (known after apply)

─────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly
these actions if you run "terraform apply" now.
tdg-zerobig-wsls#
tdg-zerobig-wsls#
tdg-zerobig-wsls# terraform apply -auto-approve
azurerm_resource_group.rg: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo]
azurerm_container_registry.zerobigaks-demo: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo]
azurerm_kubernetes_cluster.zerobigaks-demo: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo]
azurerm_role_assignment.enablePulling: Refreshing state... [id=/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo/providers/Microsoft.Authorization/roleAssignments/41ac72ed-cc01-d35e-a12c-23c90fbebfe6]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply"
which may have affected this plan:

  # azurerm_container_registry.zerobigaks-demo has been deleted
  - resource "azurerm_container_registry" "zerobigaks-demo" {
      - id                            = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo" -> null
        name                          = "tdgzeroacrdemo"
        # (16 unchanged attributes hidden)
    }

  # azurerm_kubernetes_cluster.zerobigaks-demo has been deleted
  - resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
      - fqdn                                = "zerobigaksdemo-9fctjqgz.hcp.koreacentral.azmk8s.io" -> null
      - id                                  = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo" -> null
        name                                = "tdgzeroaksdemo"
      - node_resource_group                 = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral"
 -> null
        tags                                = {
            "Environment" = "DEV"
        }
        # (21 unchanged attributes hidden)

      - kubelet_identity {
          - object_id                 = "02a2ad14-8c56-420c-ba60-e95b16680a98" -> null
            # (2 unchanged attributes hidden)
        }

        # (3 unchanged blocks hidden)
    }

  # azurerm_resource_group.rg has been deleted
  - resource "azurerm_resource_group" "rg" {
        id       = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo"
      - location = "koreacentral" -> null
      - name     = "RG-TDG-CMS-2023-AKSDemo" -> null
    }


Unless you have made equivalent changes to your configuration, or ignored the relevant attributes
using ignore_changes, the following plan may include actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_container_registry.zerobigaks-demo will be created
  + resource "azurerm_container_registry" "zerobigaks-demo" {
      + admin_enabled                 = false
      + admin_password                = (sensitive value)
      + admin_username                = (known after apply)
      + encryption                    = (known after apply)
      + export_policy_enabled         = true
      + id                            = (known after apply)
      + location                      = "koreacentral"
      + login_server                  = (known after apply)
      + name                          = "tdgzeroacrdemo"
      + network_rule_bypass_option    = "AzureServices"
      + network_rule_set              = (known after apply)
      + public_network_access_enabled = true
      + resource_group_name           = "RG-TDG-CMS-2023-AKSDemo"
      + retention_policy              = (known after apply)
      + sku                           = "Standard"
      + trust_policy                  = (known after apply)
      + zone_redundancy_enabled       = false
    }

  # azurerm_kubernetes_cluster.zerobigaks-demo will be created
  + resource "azurerm_kubernetes_cluster" "zerobigaks-demo" {
      + api_server_authorized_ip_ranges     = (known after apply)
      + dns_prefix                          = "zerobigaksdemo"
      + fqdn                                = (known after apply)
      + http_application_routing_zone_name  = (known after apply)
      + id                                  = (known after apply)
      + image_cleaner_enabled               = false
      + image_cleaner_interval_hours        = 48
      + kube_admin_config                   = (sensitive value)
      + kube_admin_config_raw               = (sensitive value)
      + kube_config                         = (sensitive value)
      + kube_config_raw                     = (sensitive value)
      + kubernetes_version                  = (known after apply)
      + location                            = "koreacentral"
      + name                                = "tdgzeroaksdemo"
      + node_resource_group                 = (known after apply)
      + oidc_issuer_url                     = (known after apply)
      + portal_fqdn                         = (known after apply)
      + private_cluster_enabled             = false
      + private_cluster_public_fqdn_enabled = false
      + private_dns_zone_id                 = (known after apply)
      + private_fqdn                        = (known after apply)
      + public_network_access_enabled       = true
      + resource_group_name                 = "RG-TDG-CMS-2023-AKSDemo"
      + role_based_access_control_enabled   = true
      + run_command_enabled                 = true
      + sku_tier                            = "Free"
      + tags                                = {
          + "Environment" = "DEV"
        }
      + workload_identity_enabled           = false

      + default_node_pool {
          + kubelet_disk_type    = (known after apply)
          + max_pods             = (known after apply)
          + name                 = "default"
          + node_count           = 2
          + node_labels          = (known after apply)
          + orchestrator_version = (known after apply)
          + os_disk_size_gb      = (known after apply)
          + os_disk_type         = "Managed"
          + os_sku               = (known after apply)
          + scale_down_mode      = "Delete"
          + type                 = "VirtualMachineScaleSets"
          + ultra_ssd_enabled    = false
          + vm_size              = "Standard_D2_v2"
          + workload_runtime     = (known after apply)
        }

      + identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
          + type         = "SystemAssigned"
        }
    }

  # azurerm_resource_group.rg will be created
  + resource "azurerm_resource_group" "rg" {
      + id       = (known after apply)
      + location = "koreacentral"
      + name     = "RG-TDG-CMS-2023-AKSDemo"
    }

  # azurerm_role_assignment.enablePulling will be created
  + resource "azurerm_role_assignment" "enablePulling" {
      + id                               = (known after apply)
      + name                             = (known after apply)
      + principal_id                     = (known after apply)
      + principal_type                   = (known after apply)
      + role_definition_id               = (known after apply)
      + role_definition_name             = "AcrPull"
      + scope                            = (known after apply)
      + skip_service_principal_aad_check = true
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  ~ aks_fqdn    = "zerobigaksdemo-9fctjqgz.hcp.koreacentral.azmk8s.io" -> (known after apply)
  ~ aks_id      = "/subscriptions/98483591-af84-4569-ba3d-0711ced77b32/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo" -> (known after apply)
  ~ aks_node_rg = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral" -> (known after apply)
azurerm_resource_group.rg: Creating...
azurerm_resource_group.rg: Creation complete after 2s [id=/subscriptions/1199b626-a317-4559-9289-caba7859ee88/resourceGroups/RG-TDG-CMS-2023-AKSDemo]
azurerm_container_registry.zerobigaks-demo: Creating...
azurerm_kubernetes_cluster.zerobigaks-demo: Creating...
azurerm_container_registry.zerobigaks-demo: Still creating... [10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [10s elapsed]
azurerm_container_registry.zerobigaks-demo: Still creating... [20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [20s elapsed]
azurerm_container_registry.zerobigaks-demo: Creation complete after 26s [id=/subscriptions/1199b626-a317-4559-9289-caba7859ee88/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [1m50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [2m50s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m0s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m10s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m20s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m30s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Still creating... [3m40s elapsed]
azurerm_kubernetes_cluster.zerobigaks-demo: Creation complete after 3m49s [id=/subscriptions/1199b626-a317-4559-9289-caba7859ee88/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo]
azurerm_role_assignment.enablePulling: Creating...
azurerm_role_assignment.enablePulling: Still creating... [10s elapsed]
azurerm_role_assignment.enablePulling: Still creating... [20s elapsed]
azurerm_role_assignment.enablePulling: Still creating... [30s elapsed]
azurerm_role_assignment.enablePulling: Creation complete after 33s [id=/subscriptions/1199b626-a317-4559-9289-caba7859ee88/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerRegistry/registries/tdgzeroacrdemo/providers/Microsoft.Authorization/roleAssignments/7cc351f4-a90e-aa84-d037-54aa37d5129f]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

aks_fqdn = "zerobigaksdemo-m2wziz05.hcp.koreacentral.azmk8s.io"
aks_id = "/subscriptions/1199b626-a317-4559-9289-caba7859ee88/resourceGroups/RG-TDG-CMS-2023-AKSDemo/providers/Microsoft.ContainerService/managedClusters/tdgzeroaksdemo"
aks_node_rg = "MC_RG-TDG-CMS-2023-AKSDemo_tdgzeroaksdemo_koreacentral"

Terraform 배포 결과 확인

Azure Portal로 이동하여 배포 결과를 확인한다.

...

ACR에 대한 AKS 관리 ID Role 부여 현황을 확인한다.

...

AKS 클러스터 검증

# Define variables
AKS_Name="tdgzeroaksdemo"

ACR_Name="tdgzeroacrdemo"

RG_Name="RG-TDG-CMS-2023-AKSDemo"

# Get AKS credentials
az aks get-credentials -g $RG_Name -n $AKS_Name

# Get Node Info
kubectl get node

Code Block
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ AKS_Name="tdgzeroaksdemo"ACR_Name="tdgzeroacrdemo"RG_Name="RG-TDG-CMS-2023-AKSDemo"
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ AKS_Name="tdgzeroaksdemo"
ACR_Name="tdgzeroacrdemo"
RG_Name="RG-TDG-CMS-2023-AKSDemo"
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ az aks get-credentials -g $RG_Name -n $AKS_Name
Merged "tdgzeroaksdemo" as current context in /home/zerobig/.kube/config
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl get node
NAME                              STATUS   ROLES   AGE   VERSION
aks-default-34060743-vmss000000   Ready    agent   22m   v1.27.7
aks-default-34060743-vmss000001   Ready    agent   22m   v1.27.7

Application 배포 및 검증

사전 작업

애플리케이션 빌드 및 구성을 위해 먼저 JAVA, Maven 설치가 필요하다.

JAVA 설치

Code Block
# Install JDK
sudo apt update
sudo apt install default-jdk

# Verify the installation
java -version

MAVEN 설치

Code Block
# Check the latest version
https://downloads.apache.org/maven/maven-3/

# Download Apache Maven
wget https://downloads.apache.org/maven/maven-3/3.9.5/binaries/apache-maven-3.9.5-bin.tar.gz -P /tmp

# Extract the Archive
sudo tar xf /tmp/apache-maven-3.9.5-bin.tar.gz -C /opt

# Create a symbolic link
sudo ln -s /opt/apache-maven-3.9.5 /opt/maven

# Setup environment variables
sudo vi /etc/profile.d/maven.sh

export JAVA_HOME=/usr/lib/jvm/default-java
export M2_HOME=/opt/maven
export MAVEN_HOME=/opt/maven
export PATH=${M2_HOME}/bin:${PATH}


sudo chmod +x /etc/profile.d/maven.sh
source /etc/profile.d/maven.sh

# Verify the installation
mvn -version

소스 준비 및 로컬 검증

새로 터미널을 띄워 wsl로 전환 후에 샘플 소스를 내려 받는다.

Code Block
git clone https://github.com/zer0big/gs-spring-boot-aks.git 
cd gs-spring-boot-aks
code .

mvn clean package 명령으로 컴파일을 수행 한다. 참고로 clean 옵션은 maven build 시 생성된 모든 것들을 삭제한다.

Code Block
mvn clean package -DskipTests

target 디렉토리로 이동 후 java -jar 명령을 통해 로컬에서 샘플 소스의 컴파일 결과를 실행하고 브라우저를 통해 유효성을 검증한다.

Code Block
java -jar gs-spring-boot-docker-0.1.0.jar
Code Block
zerobig@winvm4azuresa:~/gs-spring-boot-aks/target$ java -jar gs-spring-boot-docker-0.1.0.jar

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.0.RELEASE)

2022-07-17 17:56:30.666  INFO 7189 --- [           main] hello.Application                        : Starting Application v0.1.0 on winvm4azuresa with PID 7189 (/home/zerobig/gs-spring-boot-aks/target/gs-spring-boot-docker-0.1.0.jar started by zerobig in /home/zerobig/gs-spring-boot-aks/target)
2022-07-17 17:56:30.669  INFO 7189 --- [           main] hello.Application                        : No active profile set, falling back to default profiles: default
2022-07-17 17:56:32.043  INFO 7189 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2022-07-17 17:56:32.062  INFO 7189 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2022-07-17 17:56:32.063  INFO 7189 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.35]
2022-07-17 17:56:32.161  INFO 7189 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2022-07-17 17:56:32.161  INFO 7189 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1406 ms
2022-07-17 17:56:32.440  INFO 7189 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2022-07-17 17:56:32.657  INFO 7189 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-07-17 17:56:32.683  INFO 7189 --- [           main] hello.Application                        : Started Application in 3.058 seconds (JVM running for 3.798)
2022-07-17 17:57:39.998  INFO 7189 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-07-17 17:57:40.000  INFO 7189 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2022-07-17 17:57:40.020  INFO 7189 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 20 ms

별도의 Windows 터미널 창에서 다음 명령을 수행한다.

Code Block
start http://localhost:8080 

...

검증 완료 후 Ctrl + c를 실행하여 실행을 중지한다.

다음 작업 진행에 앞서 Bash 터미널 창에서 현재 target 디렉토리에서 상위 디렉토리로 이동한다. 

Code Block
cd ..

Containerizing 및 로컬 검증

docker build 명령을 수행하여 docker images를 생성한다.

Code Block
docker build -t appmod-demo4tdgcms .
docker images

Code Block
zerobig@sa-winvm:/mnt/c/Users/zerobig/gs-spring-boot-aks$ docker build -t appmod-demo4tdgcms .
[+] Building 10.0s (7/7) FINISHED                                                                                                            docker:default
 => [internal] load .dockerignore                                                                                                                      0.1s
 => => transferring context: 2B                                                                                                                        0.0s
 => [internal] load build definition from Dockerfile                                                                                                   0.2s
 => => transferring dockerfile: 163B                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/openjdk:8-jdk-alpine                                                                                2.8s
 => [internal] load build context                                                                                                                      1.0s
 => => transferring context: 16.47MB                                                                                                                   0.9s
 => [1/2] FROM docker.io/library/openjdk:8-jdk-alpine@sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3                          5.3s
 => => resolve docker.io/library/openjdk:8-jdk-alpine@sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3                          0.1s
 => => sha256:c2274a1a0e2786ee9101b08f76111f9ab8019e368dce1e325d3c284a0ca33397 70.73MB / 70.73MB                                                       3.4s
 => => sha256:94792824df2df33402f201713f932b58cb9de94a0cd524164a0f2283343547b3 1.64kB / 1.64kB                                                         0.0s
 => => sha256:44b3cea369c947527e266275cee85c71a81f20fc5076f6ebb5a13f19015dce71 947B / 947B                                                             0.0s
 => => sha256:a3562aa0b991a80cfe8172847c8be6dbf6e46340b759c2b782f8b8be45342717 3.40kB / 3.40kB                                                         0.0s
 => => sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10 2.76MB / 2.76MB                                                         0.6s
 => => sha256:f910a506b6cb1dbec766725d70356f695ae2bf2bea6224dbe8c7c6ad4f3664a2 238B / 238B                                                             0.3s
 => => extracting sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f10                                                              0.4s
 => => extracting sha256:f910a506b6cb1dbec766725d70356f695ae2bf2bea6224dbe8c7c6ad4f3664a2                                                              0.0s
 => => extracting sha256:c2274a1a0e2786ee9101b08f76111f9ab8019e368dce1e325d3c284a0ca33397                                                              1.4s
 => [2/2] ADD target/*.jar app.jar                                                                                                                     1.5s
 => exporting to image                                                                                                                                 0.2s
 => => exporting layers                                                                                                                                0.1s
 => => writing image sha256:e961dcb75ee02609f3339978200c57b972328ccc39d50fcd497233d2bdda8ef4                                                           0.0s
 => => naming to docker.io/library/appmod-demo4tdgcms                                                                                                  0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
zerobig@sa-winvm:/mnt/c/Users/zerobig/gs-spring-boot-aks$ docker images
REPOSITORY           TAG       IMAGE ID       CREATED          SIZE
appmod-demo4tdgcms   latest    e961dcb75ee0   10 seconds ago   121MB

docker run 명령을 수행하여 로컬에서 동작 유효성을 검증한다. 로컬포트는 임의로 지정 가능하다. 다음 예에서는 8888을 사용한다.

Code Block
zerobig@sa-winvm:/mnt/c/Users/zerobig/gs-spring-boot-aks$ docker run -d -p 8888:8080 appmod-demo4tdgcms
408c86d81b75b823858263d8f9b79a6d4ddf06d5ca6b63e04e0fa0ba431bb10a

별도의 Windows 터미널 창에서 다음 명령을 수행한다.

Code Block
start http://localhost:8888

...

Docker Tag, Push

최조 작업하던 창으로 이동한다.

$ACR_Name 변수값을 반환하여야 하며, 안되었다면 다시 변수값을 입력 한다.

자신의 ACR 서버 주소 형식에 맞게 docker tag하고 push 명령을 수행하여 생성한 ACR에 images를 Push한다.

Code Block
docker tag appmod-demo4tdgcms $ACR_Name.azurecr.io/appmod-demo4tdgcms    //자신의 ACR 서버 주소로 변경
docker images
az acr login -n $ACR_Name
docker push $ACR_Name.azurecr.io/appmod-demo4tdgcms

Code Block
tdg-zerobig-wsls# echo $ACR_Name
tdgzeroacrdemo
tdg-zerobig-wsls# docker tag appmod-demo4tdgcms $ACR_Name.azurecr.io/appmod-demo4tdgcms
tdg-zerobig-wsls#
tdg-zerobig-wsls# docker images
REPOSITORY                                     TAG       IMAGE ID       CREATED         SIZE
tdgzeroacrdemo.azurecr.io/appmod-demo4tdgcms   latest    4498a272643f   5 minutes ago   121MB
appmod-demo4tdgcms                             latest    4498a272643f   5 minutes ago   121MB
tdg-zerobig-wsls#
tdg-zerobig-wsls# az acr login -n $ACR_Name
Login Succeeded
tdg-zerobig-wsls#
tdg-zerobig-wsls# docker push $ACR_Name.azurecr.io/appmod-demo4tdgcms
Using default tag: latest
The push refers to repository [tdgzeroacrdemo.azurecr.io/appmod-demo4tdgcms]
6d14af32611d: Pushed
ceaf9e1ebef5: Pushed
9b9b7f3d56a0: Pushed
f1b5933fe4b5: Pushed
latest: digest: sha256:da72ad49804f887e6f1c7ef0a176a829e9881eb248a896117973a0d7b040c4e3 size: 1159
tdg-zerobig-wsls#

Azure Portal에서 등록 결과를 확인한다.

...

YAML Manifest 구성

k8s/deploy-svc.yaml 파일을 열어 “images” 정보를 각자의 값으로 업데이트 한다.

Code Block
apiVersion: apps/v1
kind: Deployment
metadata:
  name: zeroaksdemo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: zero-aks-app
  template:
    metadata:
      labels:
        app: zero-aks-app
    spec:
      containers:
        - name: zero-aks-demo
          image: tdgzeroacrdemo.azurecr.io/appmod-demo4tdccms:latest
          ports:
            - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
    name: zero-aks-app
spec:
  ports:
    - name: http-port
      port: 80
      targetPort: 8080
  selector:
    app: zero-aks-app
  type: LoadBalancer

YAML Manifest 배포 및 결과 검증

Code Block
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl get nodes
NAME                              STATUS   ROLES   AGE   VERSION
aks-default-34060743-vmss000000   Ready    agent   54m   v1.27.7
aks-default-34060743-vmss000001   Ready    agent   54m   v1.27.7
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl apply -f k8s/deploy-svc.yaml
deployment.apps/zeroaksdemo created
service/zero-aks-app created
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl get pod
NAME                           READY   STATUS    RESTARTS   AGE
zeroaksdemo-5686444c7c-29x5m   1/1     Running   0          10s
zeroaksdemo-5686444c7c-f2z85   1/1     Running   0          10s
zeroaksdemo-5686444c7c-mfb65   1/1     Running   0          10s
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl get svc
NAME           TYPE           CLUSTER-IP   EXTERNAL-IP      PORT(S)        AGE
kubernetes     ClusterIP      10.0.0.1     <none>           443/TCP        56m
zero-aks-app   LoadBalancer   10.0.50.60   20.196.252.253   80:32045/TCP   18s
zerobig@sa-winvm:/mnt/c/Users/zerobig/TerroformAksDeployDemo$ kubectl get pod
NAME                           READY   STATUS    RESTARTS   AGE
zeroaksdemo-5686444c7c-29x5m   1/1     Running   0          68s
zeroaksdemo-5686444c7c-f2z85   1/1     Running   0          68s
zeroaksdemo-5686444c7c-mfb65   1/1     Running   0          68s

Image Added

리소스 정리

모든 테스트를 수행하고 정상적으로 결과가 검증되었다면, terraform destroy 명령을 시행하여 배포한 리소스를 제거한다.