안녕하세요 chhanz 입니다.
이번 포스팅에서는 Kubernetes에 기반을 둔 Developer-Oriented PaaS 인 Openshift를 살펴볼 것입니다.

Openshift 란?


OpenShift 는 개발자 및 IT 운영팀을 단일 플랫폼에서 통합하여, 하이브리드 클라우드 및 멀티 클라우드 인프라 전반에서 애플리케이션을 일관되게 구축, 배포 및 관리하도록 지원하는 플랫폼입니다.

Openshift 이점


주요 이점

  • 애플리케이션 라이프사이클 전반에 걸쳐 운영 및 개발팀에서 더 큰 가치 실현
  • 애플리케이션 개발 주기 단축 및 소프트웨어 배포 빈도 증가
  • 하이브리드 클라우드 및 멀티 클라우드 전반에서 IT 운영 비용 절감 및 애플리케이션 이식성 실현

개발팀을 위한 이점

  • Openshift는 개발자에게 셀프 서비스 방식으로 애플리케이션과 컴포넌트를 프로비저닝, 빌드 및 배포하도록 지원하는 최적의 플랫폼입니다.
  • Source To Image(S2I) 프로세스 같은 자동화된 워크플로우 덕분에 소스 형상 관리의 소스 코드를 즉시 실행 가능한 도커 포맷 컨테이너 이미지로 간단하게 생성할 수 있습니다.
  • Openshift는 CI(Continuous Integration)와 CD(Continuous Delivery)의 통합 툴을 제공하기 때문에 모든 조직에게 이상적인 솔루션이 될 수 있습니다.

운영팀을 위한 이점

  • Openshift는 IT 운영팀에 애플리케이션 빌드 배포 자동화와 정책 기반 권한 관리를 지원하는 안전한 엔터프라이즈급 Kubernetes를 제공합니다.
  • 클러스터 서비스, 스케줄링 그리고 오케스트레이션을 통해 부하 분산과 자동 스케일링 기능을 제공합니다. 보안 기능을 통해 테넌트가 다른 애플리케이션이나 기본 호스트에 지장을 주지 않도록 방지합니다.
  • Openshift는 Persistent Storage를 Linux 컨테이너에 직접 연결할 수 있기 때문에 IT 조직은 하나의 플랫폼에서 스테이트풀(Stateful) 및 스테이트리스(Stateless) 애플리케이션을 모두 실행할 수 있습니다.

Openshift 종류


Openshift 에는 다양한 버전이 존재합니다.

  • OpenShift Origin
    • OpenShift Online, OpenShift Dedicated 및 OpenShift Container Platform에서 사용되는 업스트림 커뮤니티 프로젝트입니다.
  • OpenShift Container Platform
    • OpenShift Container Platform은 Red Hat이 제공하고 지원하는 엔터프라이즈 버전입니다.
  • OpenShift Online
    • OpenShift Online은 Red Hat의 호스팅형 퍼블릭 PaaS 로서 클라우드에서 애플리케이션 개발, 구축, 배포, 호스팅 솔루션을 제공합니다.
  • OpenShift Dedicated
    • OpenShift Dedicated는 퍼블릭 클라우드에서 관리형 싱글 테넌트 OpenShift 환경을 제공합니다. 전체 OpenShift Cluster를 기업 전용 솔루션으로 구축하고 Red Hat을 통해 종합적으로 관리합니다.

Openshift Origin 설치


Openshift 설치 테스트 환경은 아래와 같습니다.

CentOS Linux release 7.6.1810 (Core)
Openshift Origin v3.11
Docker v1.13.1

Openshift 를 설치하기 위해서는 기본적으로 필요한 OS 설정 및 Infra 구성이 필요합니다.

  • OS 설정
    1) SELinux : Enforcing
    2) Firewalld Disable / iptables Enable (설치간 자동으로 설정 진행됨)
    3) NetworkManager Enable

  • 추가 필요 Infra 요소
    1) Internal DNS 서버 (이번 포스팅에서는 DNS 시스템 구성 내용은 제외되어 있습니다.)

위와 같이 Openshift 를 배포하기 위해서는 기본 설정이 필요합니다.

주요 OS 설정 부분


/etc/sysconfig/network-script/ 의 내용은 아래와 같은 내부 Internal DNS 및 Upstream DNS 설정 해야됩니다.

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
NAME=eth1
UUID=XXXXXXXX-XXXXX-XXXX-XXXX-XXXXXXXXXXXXX
DEVICE=eth1
ONBOOT=yes
IPADDR=192.168.XX.50
NETMASK=255.255.255.0
GATEWAY=192.168.XX.XX
DNS1=192.168.XX.111      # Internal DNS
DNS2=1.1.1.1             # Upstream

SELinux 설정 확인

# cat /etc/selinux/config
...
SELINUX=enforcing         # Enforcing 설정
...

DNS 설정 부분


Internal DNS 는 아래와 같이 Zone File 을 설정합니다.

# cat /var/named/example.com.zone
$TTL 3H
@	IN SOA	ns.example.com. root.example.com. (
					0	; serial
					1D	; refresh
					1H	; retry
					1W	; expire
					3H )	; minimum

	IN      NS      ns.example.com.
ns	IN	A	192.168.XX.111
*	IN	A	192.168.XX.50             # wildcard DNS 설정
master	IN	A	192.168.XX.50
;

wildcard DNS 설정을 합니다.
참고자료 : https://docs.openshift.com/container-platform/3.11/install/prerequisites.html#wildcard-dns-prereq

Requirements Package 설치


Openshift 설치를 하기 위해서 centos-openshift-origin311 Repository 를 Enable 해야됩니다.

[root@master ~]# yum install centos-release-openshift-origin311.noarch
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.nara.wide.ad.jp
 * extras: data.aonenetworks.kr
 * updates: ftp.nara.wide.ad.jp
Resolving Dependencies
--> Running transaction check
---> Package centos-release-openshift-origin311.noarch 0:1-2.el7.centos will be installed
--> Processing Dependency: centos-release-paas-common for package: centos-release-openshift-origin311-1-2.el7.centos.noarch
--> Processing Dependency: centos-release-ansible26 for package: centos-release-openshift-origin311-1-2.el7.centos.noarch
--> Running transaction check
---> Package centos-release-ansible26.noarch 0:1-3.el7.centos will be installed
--> Processing Dependency: centos-release-configmanagement for package: centos-release-ansible26-1-3.el7.centos.noarch
---> Package centos-release-paas-common.noarch 0:1-1.el7.centos will be installed
--> Running transaction check
---> Package centos-release-configmanagement.noarch 0:1-1.el7.centos will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
===================================================================================================================================================================================================================================================================================================
 Package                                                                                   Arch                                                          Version                                                               Repository                                                     Size
===================================================================================================================================================================================================================================================================================================
Installing:
 centos-release-openshift-origin311                                                        noarch                                                        1-2.el7.centos                                                        extras                                                         11 k
Installing for dependencies:
 centos-release-ansible26                                                                  noarch                                                        1-3.el7.centos                                                        extras                                                        4.1 k
 centos-release-configmanagement                                                           noarch                                                        1-1.el7.centos                                                        extras                                                        4.3 k
 centos-release-paas-common                                                                noarch                                                        1-1.el7.centos                                                        extras                                                         11 k
 
Transaction Summary
===================================================================================================================================================================================================================================================================================================
Install  1 Package (+3 Dependent packages)
 
Total download size: 31 k
Installed size: 39 k
Is this ok [y/d/N]:y
 
...
Running transaction
  Installing : centos-release-configmanagement-1-1.el7.centos.noarch                                                                                                                                                                                                                           1/4
  Installing : centos-release-ansible26-1-3.el7.centos.noarch                                                                                                                                                                                                                                  2/4
  Installing : centos-release-paas-common-1-1.el7.centos.noarch                                                                                                                                                                                                                                3/4
  Installing : centos-release-openshift-origin311-1-2.el7.centos.noarch                                                                                                                                                                                                                        4/4
  Verifying  : centos-release-paas-common-1-1.el7.centos.noarch                                                                                                                                                                                                                                1/4
  Verifying  : centos-release-configmanagement-1-1.el7.centos.noarch                                                                                                                                                                                                                           2/4
  Verifying  : centos-release-openshift-origin311-1-2.el7.centos.noarch                                                                                                                                                                                                                        3/4
  Verifying  : centos-release-ansible26-1-3.el7.centos.noarch                                                                                                                                                                                                                                  4/4
 
Installed:
  centos-release-openshift-origin311.noarch 0:1-2.el7.centos
 
Dependency Installed:
  centos-release-ansible26.noarch 0:1-3.el7.centos                                             centos-release-configmanagement.noarch 0:1-1.el7.centos                                             centos-release-paas-common.noarch 0:1-1.el7.centos
 
Complete!
[root@master ~]#

위와 같이 설치가 완료가 되면 추가로 Openshift Origin v3.11 설치를 위한 Repogitory 가 Enable 됩니다.

[root@master ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.nara.wide.ad.jp
 * centos-ansible26: data.aonenetworks.kr
 * extras: data.aonenetworks.kr
 * updates: ftp.nara.wide.ad.jp
centos-ansible26                                                                                                                                                                                                                                                            | 2.9 kB  00:00:00
centos-openshift-origin311                                                                                                                                                                                                                                                  | 2.9 kB  00:00:00
(1/2): centos-ansible26/7/x86_64/primary_db                                                                                                                                                                                                                                 | 6.5 kB  00:00:00
(2/2): centos-openshift-origin311/primary_db                                                                                                                                                                                                                                |  19 kB  00:00:00
repo id                                                                                                                                         repo name                                                                                                                                    status
base/7/x86_64                                                                                                                                   CentOS-7 - Base                                                                                                                              10,019
centos-ansible26/7/x86_64                                                                                                                       CentOS-7 - Ansible26                                                                                                                              8
centos-openshift-origin311                                                                                                                      CentOS OpenShift Origin                                                                                                                          31
extras/7/x86_64                                                                                                                                 CentOS-7 - Extras                                                                                                                               409
updates/7/x86_64                                                                                                                                CentOS-7 - Updates                                                                                                                            1,982
repolist: 12,449
[root@master ~]#

Openshift Origin v3.11 은 ansible Version 에 종속성이 있어서 추가로 centos-ansible26 Repository 가 Enable 됩니다.

추가로 Openshift ansible Playbook 및 ansible 을 설치합니다.

[root@master ~]# yum install openshift-ansible ansible

설치가 완료되면 아래와 같이 /usr/share/ansible/openshift-ansible/ 경로에 Playbook 이 생성됩니다.

[root@master ~]# cd /usr/share/ansible/openshift-ansible/
[root@master openshift-ansible]# ls -la
합계 12
drwxr-xr-x.  5 root root   72  6월  5 07:14 .
drwxr-xr-x.  3 root root   31  6월  5 07:14 ..
-rw-r--r--.  1 root root 1303 11월  2  2018 ansible.cfg
drwxr-xr-x.  3 root root   34  6월  5 07:15 inventory
drwxr-xr-x. 38 root root 4096  6월  5 07:14 playbooks
drwxr-xr-x. 93 root root 4096  6월  5 07:14 roles
[root@master openshift-ansible]#

All-in-one Inventory 작성


노드 하나에 master, infra, compute 의 기능을 전부 설치하는 All-in-one Inventory 를 작성합니다.

[root@master openshift-ansible]# cat inventory/hosts.localhost
#bare minimum hostfile
 
[OSEv3:children]
masters
nodes
etcd
 
[OSEv3:vars]
# if your target hosts are Fedora uncomment this
#ansible_python_interpreter=/usr/bin/python3
openshift_deployment_type=origin
openshift_portal_net=172.30.0.0/16
# localhost likely doesn't meet the minimum requirements
openshift_disable_check=disk_availability,memory_availability
 
openshift_node_groups=[{'name': 'node-config-all-in-one', 'labels': ['node-role.kubernetes.io/master=true', 'node-role.kubernetes.io/infra=true', 'node-role.kubernetes.io/compute=true']}]
openshift_release="3.11"
openshift_master_default_subdomain=apps.example.com
 
# uncomment the following to enable htpasswd authentication; defaults to AllowAllPasswordIdentityProvider
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
 
 
[masters]
localhost ansible_connection=local
 
[etcd]
localhost ansible_connection=local
 
[nodes]
# openshift_node_group_name should refer to a dictionary with matching key of name in list openshift_node_groups.
localhost ansible_connection=local openshift_node_group_name="node-config-all-in-one"

openshift_master_identity_providers 항목을 추가하여 htpasswd 기능을 활성화 하였습니다.

prerequisites.yml 배포


Openshift Cluster 배포하기 전에 prerequisites.yml 를 실행하여 Cluster 구성간 필요한 설정 작업 및 추가 패키지 설치를 합니다.

# ansible-playbook -i inventory/hosts.localhost playbooks/prerequisites.yml
...

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=83   changed=23   unreachable=0    failed=0
 
 
INSTALLER STATUS **********************************************************************************************************************************************************************************************************************************************************************************
Initialization  : Complete (0:00:35)
Tuesday 04 June 2019  10:38:27 +0900 (0:00:00.032)       0:01:35.916 ***********
===============================================================================
Ensure openshift-ansible installer package deps are installed ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 19.40s
container_runtime : Install Docker -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.40s
os_firewall : need to pause here, otherwise the iptables service starting can sometimes cause ssh to fail --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.13s
os_firewall : Wait 10 seconds after disabling firewalld ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.10s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.49s
openshift_excluder : Install docker excluder - yum ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.01s
os_firewall : Install iptables packages ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.08s
container_runtime : restart container runtime ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.88s
container_runtime : Fixup SELinux permissions for docker ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.54s
openshift_repos : refresh cache ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.76s
openshift_repos : Ensure libselinux-python is installed ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.74s
Gather Cluster facts ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.66s
os_firewall : Ensure firewalld service is not enabled -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.60s
openshift_repos : Configure origin gpg keys ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.60s
os_firewall : Start and enable iptables service -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.55s
container_runtime : Get current installed Docker version ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.52s
container_runtime : Place additional/blocked/insecure registries in /etc/containers/registries.conf ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.50s
openshift_repos : Configure correct origin release repository ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.46s
container_runtime : Configure Docker service unit file ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.45s
Detecting Operating System from ostree_booted ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.44s
[root@master openshift-ansible]#

deploy_cluster.yml 배포


prerequisites.yml 가 위와 같이 정상적으로 배포가 되면 Cluster 구성 준비가 완료 된 것입니다.
deploy_cluster.yml 을 이용하여 Cluster 배포를 시작합니다.

# ansible-playbook -i inventory/hosts.localhost playbooks/deploy_cluster.yml
...

PLAY RECAP ****************************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=566  changed=138  unreachable=0    failed=0
 
 
INSTALLER STATUS **********************************************************************************************************************************************************************************************************************************************************************************
Initialization               : Complete (0:00:23)
Health Check                 : Complete (0:00:38)
Node Bootstrap Preparation   : Complete (0:01:14)
etcd Install                 : Complete (0:00:32)
Master Install               : Complete (0:04:37)
Master Additional Install    : Complete (0:00:36)
Node Join                    : Complete (0:00:18)
Hosted Install               : Complete (0:00:40)
Cluster Monitoring Operator  : Complete (0:00:11)
Web Console Install          : Complete (0:00:40)
Console Install              : Complete (0:00:18)
metrics-server Install       : Complete (0:00:01)
Service Catalog Install      : Complete (0:03:39)
Tuesday 04 June 2019  11:23:04 +0900 (0:00:00.096)       0:14:35.316 **********
===============================================================================
openshift_control_plane : Wait for all control plane pods to become ready ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 97.17s
/root/openshift-ansible/roles/openshift_control_plane/tasks/main.yml:272 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
template_service_broker : Verify that TSB is running -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 53.92s
/root/openshift-ansible/roles/template_service_broker/tasks/deploy.yml:52 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_service_catalog : Wait for Controller Manager rollout success ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 43.92s
/root/openshift-ansible/roles/openshift_service_catalog/tasks/start.yml:14 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_control_plane : Wait for control plane pods to appear --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 42.90s
/root/openshift-ansible/roles/openshift_control_plane/tasks/main.yml:220 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Run health checks (install) - EL ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 37.39s
/root/openshift-ansible/playbooks/openshift-checks/private/install.yml:24 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_service_catalog : Wait for API Server rollout success --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 32.19s
/root/openshift-ansible/roles/openshift_service_catalog/tasks/start.yml:2 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_web_console : Pause for the web console deployment to start --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 30.16s
/root/openshift-ansible/roles/openshift_web_console/tasks/install.yml:158 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_service_catalog : oc_process ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 16.35s
/root/openshift-ansible/roles/openshift_service_catalog/tasks/install.yml:44 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_console : Waiting for console rollout to complete -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 9.06s
/root/openshift-ansible/roles/openshift_console/tasks/start.yml:2 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_manageiq : Configure role/user permissions --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.73s
/root/openshift-ansible/roles/openshift_manageiq/tasks/main.yaml:45 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuned : Restart tuned service -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.41s
/root/openshift-ansible/roles/tuned/tasks/main.yml:38 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_control_plane : Wait for APIs to become available -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 5.29s
/root/openshift-ansible/roles/openshift_control_plane/tasks/check_master_api_is_ready.yml:2 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_node : Install node, clients, and conntrack packages ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.55s
/root/openshift-ansible/roles/openshift_node/tasks/install.yml:2 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
tuned : Restart tuned service -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.05s
/root/openshift-ansible/roles/tuned/tasks/main.yml:38 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.86s
/root/openshift-ansible/playbooks/init/basic_facts.yml:2 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_excluder : Install docker excluder - yum ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.62s
/root/openshift-ansible/roles/openshift_excluder/tasks/install.yml:9 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_cli : Install clients ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 3.57s
/root/openshift-ansible/roles/openshift_cli/tasks/main.yml:2 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ansible_service_broker : Create custom resource definitions for asb ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 3.29s
/root/openshift-ansible/roles/ansible_service_broker/tasks/install.yml:128 -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_hosted : Create OpenShift router ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.22s
/root/openshift-ansible/roles/openshift_hosted/tasks/router.yml:85 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
openshift_control_plane : Start and enable self-hosting node ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.04s
/root/openshift-ansible/roles/openshift_control_plane/tasks/main.yml:201 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@master openshift-ansible]#

배포가 완료되면 아래와 같이 # oc get all 명령을 통해 Openshift 가 정상적으로 설치가 된 것을 확인 할 수 있습니다.

[root@master ~]# oc get all
NAME                           READY     STATUS    RESTARTS   AGE
pod/docker-registry-1-9t2zp    1/1       Running   0          10m
pod/registry-console-1-n4v9t   1/1       Running   0          10m
pod/router-1-58cpz             1/1       Running   3          10m

NAME                                       DESIRED   CURRENT   READY     AGE
replicationcontroller/docker-registry-1    1         1         1         10m
replicationcontroller/registry-console-1   1         1         1         10m
replicationcontroller/router-1             1         1         1         10m

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE
service/docker-registry    ClusterIP   172.30.205.4     <none>        5000/TCP                  10m
service/kubernetes         ClusterIP   172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     10m
service/registry-console   ClusterIP   172.30.204.108   <none>        9000/TCP                  10m
service/router             ClusterIP   172.30.237.203   <none>        80/TCP,443/TCP,1936/TCP   10m

NAME                                                  REVISION   DESIRED   CURRENT   TRIGGERED BY
deploymentconfig.apps.openshift.io/docker-registry    1          1         1         config
deploymentconfig.apps.openshift.io/registry-console   1          1         1         config
deploymentconfig.apps.openshift.io/router             1          1         1         config

NAME                                        HOST/PORT                                   PATH      SERVICES           PORT      TERMINATION   WILDCARD
route.route.openshift.io/docker-registry    docker-registry-default.apps.example.com              docker-registry    <all>     passthrough   None
route.route.openshift.io/registry-console   registry-console-default.apps.example.com             registry-console   <all>     passthrough   None
[root@master ~]#

Htpasswd 설정 && USER 권한 설정


htpasswd 설정을 동해 Openshift Cluster를 관리할 admin 계정을 생성합니다.

# cat /etc/origin/master/master-config.yaml | grep htpasswd
    name: htpasswd_auth
      file: /etc/origin/master/htpasswd

Master 설정을 확인하여 htpasswd 가 저장되는 경로를 확인합니다.

[root@master master]# htpasswd /etc/origin/master/htpasswd admin
New password:
Re-type new password:
Updating password for user admin

[root@master master]# cat /etc/origin/master/htpasswd
admin:$apr1$vfGx9xr2$G9/SgRxXUupu/mUVSuwmR/
[root@master master]#

htpasswd 명령을 이용하여 신규 사용자 admin 을 생성합니다.

[root@master master]# oc login -u admin
Authentication required for https://master.example.com:8443 (openshift)
Username: admin
Password:
Login successful.

You have access to the following projects and can switch between them with 'oc project <projectname>':
...
  * sample-project
...
Using project "".
[root@master master]# oc new-project sample-project
Now using project "sample-project" on server "https://master.example.com:8443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git

to build a new example application in Ruby.
[root@master master]#

신규로 생성된 admin 으로 login을 하고 사용할 Project 를 생성합니다.

[root@master master]# oc login -u system:admin
Logged into "https://master.example.com:8443" as "system:admin" using existing credentials.

You have access to the following projects and can switch between them with 'oc project <projectname>':
...
  * sample-project
...
Using project "sample-project".

[root@master master]# oc adm policy add-scc-to-user anyuid -z default
scc "anyuid" added to: ["system:serviceaccount:sample-project:default"]

[root@master master]# oc login -u admin
Logged into "https://master.example.com:8443" as "admin" using existing credentials.

You have access to the following projects and can switch between them with 'oc project <projectname>':
...
  * sample-project
...
Using project "sample-project".
[root@master master]#

default serviceaccount 에 anyuid scc 를 추가하여 pod 를 제어 할 수 있는 권한을 부여합니다.

Sample APP 배포


Sample APP 를 배포하여 실제로 Openshift 를 이용하여 서비스를 시작해보겠습니다.

S2I 기능을 이용한 APP 배포


Openshift 의 S2I 기능을 이용하여 Sample APP 를 배포하겠습니다.

APP : wildfly:latest
Source : https://github.com/chhanz/openshift-deploy.git

[root@master ~]# oc new-app wildfly~https://github.com/chhanz/openshift-deploy --name sampleapp
--> Found image 05e5cf6 (2 weeks old) in image stream "openshift/wildfly" under tag "13.0" for "wildfly"

    WildFly 13.0.0.Final
    --------------------
    Platform for building and running JEE applications on WildFly 13.0.0.Final

    Tags: builder, wildfly, wildfly13

    * A source build using source code from https://github.com/chhanz/openshift-deploy will be created
      * The resulting image will be pushed to image stream tag "sampleapp:latest"
      * Use 'start-build' to trigger a new build
    * This image will be deployed in deployment config "sampleapp"
    * Port 8080/tcp will be load balanced by service "sampleapp"
      * Other containers can access this service through the hostname "sampleapp"

--> Creating resources ...
    imagestream.image.openshift.io "sampleapp" created
    buildconfig.build.openshift.io "sampleapp" created
    deploymentconfig.apps.openshift.io "sampleapp" created
    service "sampleapp" created
--> Success
    Build scheduled, use 'oc logs -f bc/sampleapp' to track its progress.
    Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
     'oc expose svc/sampleapp'
    Run 'oc status' to view your app.

신규 APP 가 생성이 되고 sample.war 소스를 기반으로

[root@master ~]# oc logs -f bc/sampleapp
Cloning "https://github.com/chhanz/openshift-deploy" ...
	Commit:	7c4656f9d88fe9ffbf846ba0dff5b74742f34c67 (Change Location)
	Author:	chhanz <han0495@gmail.com>
	Date:	Fri Jun 7 22:00:34 2019 +0900
Using docker-registry.default.svc:5000/openshift/wildfly@sha256:895e0a6c732f8244ce75b376651155fdef12311df0d940ce111756ef43aa2bfc as the s2i builder image
Moving binaries in source directory into /wildfly/standalone/deployments for later deployment...
Moving all war artifacts from /opt/app-root/src/. directory into /wildfly/standalone/deployments for later deployment...
'/opt/app-root/src/./sample.war' -> '/wildfly/standalone/deployments/sample.war'
Moving all ear artifacts from /opt/app-root/src/. directory into /wildfly/standalone/deployments for later deployment...
Moving all rar artifacts from /opt/app-root/src/. directory into /wildfly/standalone/deployments for later deployment...
Moving all jar artifacts from /opt/app-root/src/. directory into /wildfly/standalone/deployments for later deployment...
...done

Pushing image docker-registry.default.svc:5000/sample-project/sampleapp:latest ...
Pushed 2/13 layers, 15% complete
Pushed 3/13 layers, 23% complete
Pushed 4/13 layers, 31% complete
Pushed 5/13 layers, 39% complete
Pushed 6/13 layers, 46% complete
Pushed 7/13 layers, 54% complete
Pushed 8/13 layers, 62% complete
Pushed 9/13 layers, 69% complete
Pushed 10/13 layers, 77% complete
Pushed 11/13 layers, 85% complete
Pushed 12/13 layers, 92% complete
Pushed 13/13 layers, 100% complete
Push successful

위와 같이 git 에서 소스를 가져와서 이미지로 생성하고 Openshift registory 에 Push 합니다.

[root@master ~]# oc get po
NAME                READY     STATUS      RESTARTS   AGE
sampleapp-1-build   0/1       Completed   0          4m
sampleapp-1-rsklv   1/1       Running     0          3m
[root@master ~]#

[root@master ~]# oc get svc
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
sampleapp   ClusterIP   172.30.37.118   <none>        8080/TCP   4m
[root@master ~]#

생성된 service 를 이용하여 route 를 생성합니다.

[root@master ~]# oc expose service/sampleapp
route.route.openshift.io/sampleapp exposed
[root@master ~]# oc get route
NAME        HOST/PORT                                   PATH      SERVICES    PORT       TERMINATION   WILDCARD
sampleapp   sampleapp-sample-project.apps.example.com             sampleapp   8080-tcp                 None
[root@master ~]#

sampleapp 에 대한 route 가 생성이 되었습니다.
실제로 서비스가 되는지 확인하도록 하겠습니다.


sample 서비스 정상 구동중

Web Console


Openshift 는 Web Console 을 제공하여 좀더 편리하게 서비스를 배포하고 운영 할 수 있도록 하고 있습니다.

Openshift Web Condole admin 계정으로 로그인을 하겠습니다.

배포된 sampleapp 어플리케이션 위와 같이 배포된 어플리케이션들에 대해 상태를 확인하고 Scale Up/Down 등의 기능을 수행이 가능합니다.

Openshift Grafana Openshift Origin v3.11에는 기본적으로 Prometheus + Grafana Dashboard 를 제공하고 있습니다.
(openshift-monitoring Project 에서 실행중)

이번 포스팅에서는 Openshift Origin 설치 및 APP 를 배포하여 서비스가 동작되는 것을 확인 하였습니다.
이후 포스팅에서는 Web Console 를 통한 배포/관리, HPA 를 이용한 Auto-Scaling 구현을 작성해보도록 하겠습니다.

감사합니다.

참고자료


chhanz's profile image

chhanz

2019-06-07

Read more posts by this author