BACKEND/Docker & Kubernetes

Docker Engine, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (2)

gngsn 2022. 1. 20. 22:51

๐Ÿ“Œ  Docker Series

Docker Engine, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (1) -  docker engine deep dive

> Docker Engine, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (2) - namespace, cgroup

Docker Network, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (1) - libnetwork

Docker Network, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (2) - bridge, host, none drivers

Docker, ์ œ๋Œ€๋กœ ์‚ฌ์šฉํ•˜๊ธฐ - Commands

 

 

์•ˆ๋…•ํ•˜์„ธ์š”. ์˜ค๋Š˜์€ Docker Engine, ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๊ธฐ (1) ์— ์ด์–ด์„œ,

Container ๊ธฐ์ˆ ์ด ๊ฐ€๋Šฅํ•œ ์ด์œ ์ธ ๋ฆฌ๋ˆ…์Šค ์ปค๋„์˜ namespaces, cgroups์— ๋Œ€ํ•ด ๋‹ค๋ฃจ๋ ค๊ณ  ํ•ฉ๋‹ˆ๋‹ค.

 

ํ•ด๋‹น ํฌ์ŠคํŒ…์˜ ๋ชฉ์ ์€ dockerEngine์˜ ๋‚ด๋ถ€ ๊ตฌ์กฐ์™€ ๋™์ž‘์›๋ฆฌ ๋“ฑ์„ ์ตœ๋Œ€ํ•œ ์ž์„ธํžˆ ๋‹ค๋ฃจ๋Š”๋ฐ ์žˆ์Šต๋‹ˆ๋‹ค.

์ด ๊ธ€์„ ์ฝ๋Š” ๋…์ž๊ฐ€ Docker image, container ๋“ฑ ๊ธฐ๋ณธ์ ์ธ ๊ฐœ๋…์„ ์ดํ•ดํ•œ๋‹ค๋Š” ๊ฐ€์ •ํ•˜์— ์ž‘์„ฑํ•ฉ๋‹ˆ๋‹ค.

 

 

ํ•ด๋‹น ํฌ์ŠคํŒ…์˜ ๋ชจ๋“  ๊ทธ๋ฆผ์€ ์ง์ ‘ ๊ทธ๋ฆฐ ๊ฒƒ์œผ๋กœ, ์‚ฌ์šฉ์‹œ ์ถœ์ฒ˜ ํ‘œ์‹œ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค ๐Ÿ™๐Ÿป.

-----------------    INDEX     -----------------

 

[ 1๋ถ€ ]

History

Docker Engine

 

[ 2๋ถ€ ]

Namespace

cgroups

 

----------------------------------------------

 

 

 

์ง€๋‚œ ํฌ์ŠคํŒ…์—์„œ LXC๋ฅผ ์‚ฌ์šฉํ•˜๋˜ ์˜ˆ์ „์˜ Docker์—์„œ Linux ์˜์กด์„ฑ์„ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด libcontainer๋ฅผ ์ œ์ž‘ํ–ˆ๋‹ค๊ณ  ํ–ˆ์Šต๋‹ˆ๋‹ค.

์ด๋กœ ์ธํ•ด Docker๊ฐ€ LXC๋‚˜ ์™ธ๋ถ€ ํŒจํ‚ค์ง€์— ์˜์กดํ•˜์ง€ ์•Š๊ณ ,

libcontainer๋ฅผ ํ†ตํ•ด ์ปค๋„์˜ container API๋“ค์„ ์ง์ ‘ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๊ฒŒ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

 

์ฆ‰, libcontainer๊ฐ€  ์ด์ œ๋ถ€ํ„ฐ ์•Œ์•„๋ณผ ์ปค๋„ ์„œ๋น„์Šค๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ์‹œ์Šคํ…œ์˜ ๋ฆฌ์†Œ์Šค๋ฅผ namespaces, control groups, capabilities, apparmor profiles, network interfaces, firewalling rules ๋“ฑ ์„ ์กฐ์ž‘ํ•  ์ˆ˜ ์žˆ๊ฒŒ ๋œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.

 

๊ฒฐ๋ก ์ ์œผ๋กœ, ์•„๋ž˜์™€ ๊ฐ™์€ ๊ตฌ์กฐ๋ฅผ ๋ณด์ž…๋‹ˆ๋‹ค.

 

container - namespaces + cgroups

 

์•„๋ž˜์—์„œ ๊ฐœ๋…์„ ํ™•์ธํ•ด๋ณธ ํ›„ ๊ทธ๋ฆผ์„ ๋‹ค์‹œ ๋ณด๋ฉด ์‰ฝ๊ฒŒ ์ดํ•ด๋˜์‹ค๊ฑฐ์—์š”.

๊ทธ๋Ÿผ ์ž์„ธํžˆ ์•Œ์•„๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

 

 

 

๐Ÿ—„  namespaces

Docker๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด์„œ Process๋‚˜ Network ๋“ฑ์ด ์–ด๋–ป๊ฒŒ ๊ฐ ์ปจํ…Œ์ด๋„ˆ๋งˆ๋‹ค ๋”ฐ๋กœ ๊ด€๋ฆฌ๋˜๋Š”์ง€ ๊ถ๊ธˆํ•˜์ง€ ์•Š์œผ์…จ๋‚˜์š”?

VM์—์„œ๋Š” ๊ฐ ๊ฒŒ์ŠคํŠธ ๋จธ์‹ ๋ณ„๋กœ ๋…๋ฆฝ์ ์ธ ๊ณต๊ฐ„์„ ์ œ๊ณตํ•˜๊ณ  ์„œ๋กœ๊ฐ€ ์ถฉ๋Œํ•˜์ง€ ์•Š๋„๋ก ํ•˜๋Š” ๊ธฐ๋Šฅ์„ ๊ฐ–๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. Docker์—์„œ๋Š” namespaces๋ฅผ ํ†ตํ•ด ์ด๋Ÿฌํ•œ ๋…๋ฆฝ๋œ ๊ณต๊ฐ„์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

 

namespaces์€ nested process tree๋ฅผ ๋งŒ๋“ค ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ฃผ๋ฉฐ,

์ด ๋ง์€ ๊ฐ ํ”„๋กœ์„ธ์Šค๊ฐ€ ์‹œ์Šคํ…œ ๋ฆฌ์†Œ์Šค(process IDs, hostnames, user IDs, network access, interprocess communication, filesystem ๋“ฑ)์™€ ํ•จ๊ป˜ ๊ณ ์œ ํ•˜๊ฒŒ ๋ถ„๋ฆฌ๋œ ํ”„๋กœ์„ธ์Šค ํŠธ๋ฆฌ๋ฅผ ๊ฐ€์งˆ ์ˆ˜ ์žˆ์Œ์„ ์˜๋ฏธํ•ฉ๋‹ˆ๋‹ค.

๋ถ„๋ฆฌ๋œ process tree๋Š” ๋‹ค๋ฅธ process tree์—์„œ ํ™•์ธinspectํ•˜๊ฑฐ๋‚˜ ์‚ญ์ œkill ํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.

 

PID namespace๋ฅผ ์•Œ์•„๋ณด๋ฉฐ ์ดํ•ด๋ฅผ ๋”ํ•ด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

 

 

PID namespace

 

 

๋ชจ๋“  ์‹œ์Šคํ…œ์€ ๋ถ€ํŒ… ์‹œ PID 1 ํ”„๋กœ์„ธ์Šค๊ฐ€ ์‹œ์ž‘๋˜๊ณ  ๊ทธ ์•„๋ž˜์— ๋ชจ๋“  ํ”„๋กœ์„ธ์Šค๋“ค์ด ์‹œ์ž‘๋ฉ๋‹ˆ๋‹ค.

ํ”„๋กœ์„ธ์Šค ํŠธ๋ฆฌ ๊ตฌ์กฐ

 

์ด๋•Œ, PID namespace๋กœ ๊ฒฉ๋ฆฌ๋ฅผ ํ•˜๊ฒŒ ๋˜๋ฉด

ํ•˜์œ„ namespace์˜ ํ”„๋กœ์„ธ์Šค๊ฐ€ ์ƒ์œ„ ํ”„๋กœ์„ธ์Šค์˜ ์กด์žฌ๋ฅผ ์•Œ ์ˆ˜ ์—†๊ฒŒ ๋ฉ๋‹ˆ๋‹ค. 

ํ•˜์ง€๋งŒ ์ƒ์œ„ namespace์˜ ํ”„๋กœ์„ธ์Šค๋Š” ๋‹ค๋ฅธ ํ”„๋กœ์„ธ์Šค์ธ ๊ฒƒ์ฒ˜๋Ÿผ ํ•˜์œ„ namespace์˜ ํ”„๋กœ์„ธ์Šค๋ฅผ ์ „์ฒด์ ์œผ๋กœ ๋ณผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

 

๊ทธ๋ฆผ์—์„œ ๋ณด์ด๋“ฏ์ด, PID namespace ๊ฒฉ๋ฆฌ๋ฅผ ํ†ตํ•ด

์ƒˆ๋กœ ์ƒ์„ฑ๋œ ํ•˜์œ„ ํ”„๋กœ์„ธ์Šค๋Š” ํ•˜๋‚˜์˜ ์‹œ์Šคํ…œ์ธ ๊ฒƒ์ฒ˜๋Ÿผ pid๋ฅผ 1๋กœ ๊ฐ€์ •ํ•˜์—ฌ ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค.

 

 

์ฝ”๋“œ๋ฅผ ํ†ตํ•ด namespaces๋ฅผ ํ†ตํ•œ ๊ฒฉ๋ฆฌ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•˜๊ณ ,

์–ด๋–ค namespaces๊ฐ€ ์ง€์›๋˜๋Š”์ง€ ํ™•์ธํ•ด๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

 

 

โœ”๏ธ namespace isolation

System call - clone() ์— ์ •์˜๋œ flags๋ฅผ ์ง€์ •ํ•˜์—ฌ ๋‹ค์–‘ํ•œ namespace๋ฅผ ๊ฐ–๋Š” ํ•˜์œ„ ํ”„๋กœ์„ธ์Šค๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

 

#define _GNU_SOURCE             /* See feature_test_macros(7) */
#include <sched.h>

int clone(int (*fn)(void *), void *child_stack,
          int flags, void *arg, ...
          /* pid_t *ptid, struct user_desc *tls, pid_t *ctid */ );

 

์„ธ ๋ฒˆ์งธ ์ธ์ž๋กœ flags๋ฅผ ์ง€์ •ํ•  ์ˆ˜ ์žˆ๋Š”๋ฐ์š”. 

์–ด๋–ค flags๊ฐ€ ์žˆ๊ณ , ์–ด๋–ค ์˜๋ฏธ๋ฅผ ๊ฐ€์ง€๋Š”์ง€ ํ™•์ธํ•ด๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

 

pid_t = pid = clone(cb, *stack, 
                CLONE_NEWPID |    // create new PID namespace
                CLONE_NEWNET |    // create new network namespace
                CLONE_NEWNS |     // create new mount namespace
                CLONE_NEWUTS |    // create new UTS namespace
                CLONE_NEWIPC |    // create new IPC namespace
                SIGCLD, 
              ...);

 

flags๋“ค์— ์ง€์ •๋˜๋Š” ์˜ต์…˜์€ ์•„๋ž˜์™€ ๊ฐ™์€ ์˜๋ฏธ๋ฅผ ๊ฐ–์Šต๋‹ˆ๋‹ค.

 

PID namespace

for process isolation.

PID (Process ID)๋ฅผ ๋ถ„ํ• ํ•ฉ๋‹ˆ๋‹ค.

 

NET namespace 

for managing network interfaces.

Network interface, iptables ๋“ฑ network ๋ฆฌ์†Œ์Šค์™€ ๊ด€๋ จ๋œ ์ •๋ณด๋ฅผ ๋ถ„ํ• ํ•ฉ๋‹ˆ๋‹ค.

 

IPC namespace 

for managing access to IPC resources.

Inter-process communication - ํ”„๋กœ์„ธ์Šค๊ฐ„ ํ†ต์‹ ์„ ๊ฒฉ๋ฆฌํ•ฉ๋‹ˆ๋‹ค.

 

MNT namespace 

for managing filesystem mount points.

file system์˜ mount ์ง€์ ์„ ๋ถ„ํ• ํ•˜์—ฌ ๊ฒฉ๋ฆฌํ•ฉ๋‹ˆ๋‹ค.

 

UTS namespace

for isolating kernel and version identifiers. 

hostname, domainname ์„ ๋ณ€๊ฒฝํ•˜๊ณ  ๋ถ„ํ• ํ•ฉ๋‹ˆ๋‹ค.

์ž์„ธํžˆ ๋งํ•˜์ž๋ฉด, Linux system call ์ค‘ ํ•˜๋‚˜์ธ uname์—์„œ utsname ๋ผ๋Š” struct์— ์ •์˜๋œ ์‹๋ณ„์ž ์ค‘ nodename์„ isolateํ•ฉ๋‹ˆ๋‹ค.

 

 

namespace๋กœ ์‹œ์Šคํ…œ์—์„œ ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋ถ„๋ฆฌํ•˜์—ฌ ์„œ๋น„์Šค๋ฅผ ์ œ๊ณตํ•  ์ˆ˜ ์žˆ์Œ์„ ์•Œ์•„๋ณด์•˜๋Š”๋ฐ์š”,

๋˜ ํ•˜๋‚˜ ๊ถ๊ธˆ์ฆ์ด ์ƒ๊น๋‹ˆ๋‹ค.

 

๋ถ„๋ฆฌ๋œ ํ™˜๊ฒฝ์„ ์–ด๋–ป๊ฒŒ ์‹ ๋ขฐ์„ฑ์žˆ๊ฒŒ ์‚ฌ์šฉํ•˜๊ณ , ๋ถ„๋ฆฌ๋œ ์ž์› ๊ด€๋ฆฌ๋ฅผ ํšจ์œจ์ ์œผ๋กœ ํ•  ์ˆ˜ ์žˆ์—ˆ์„์ง€ ๊ถ๊ธˆํ•˜์ง€ ์•Š์œผ์‹ ๊ฐ€์š” ? 

ํ˜ธ๊ธฐ์‹ฌ ๊ฐ•์š” ์ค‘์ธ๊ฐ€์š”..?

๋ฐ˜๊ฐ•์ œ์ ์ธ ์ด ์งˆ๋ฌธ์— ๋Œ€ํ•œ ๋‹ต๋ณ€์€, cgroups์„ ํ†ตํ•ด ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค ~

 

 

 

๐Ÿฅ  cgroups

cgroups์€ control group์„ ์˜๋ฏธํ•ฉ๋‹ˆ๋‹ค.

๊ธฐ๋ณธ์ ์œผ๋กœ ํ”„๋กœ์„ธ์Šค ๊ทธ๋ฃน๋ณ„๋กœ ์‹œ์Šคํ…œ์˜ ๋ฌผ๋ฆฌ์ ์ธ ๋ฆฌ์†Œ์Šค(hardware resources)๋ฅผ ์ œํ•œ์ด๋‚˜ ์ œ์•ฝ ์กฐ๊ฑด์„ ์ ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ œํ•œํ•  ์ˆ˜ ์žˆ๋Š” ๋ฆฌ์†Œ์Šค์—๋Š” ์•„๋ž˜์™€ ๊ฐ™์€ ๊ฒƒ๋“ค์ด ์žˆ์Šต๋‹ˆ๋‹ค.

 

  • CPU
  • Memory
  • Desk I/O
  • Network
  • Device

 

Kernel์—์„œ cgroup์„ ์ œ์–ดํ•  ๋•Œ, ๋”ฐ๋กœ system call์„ ์ œ๊ณตํ•˜์ง€ ์•Š๊ณ 

cgroupfs๋ผ๊ณ  ๋ถˆ๋ฆฌ๋Š” ์œ ์‚ฌ ํŒŒ์ผ ์‹œ์Šคํ…œpseudo-filesystem์„ ํ†ตํ•ด ์ œ๊ณต๋ฉ๋‹ˆ๋‹ค.

 

ํŒŒ์ผ์„ ํ†ตํ•ด์„œ ์ œํ•œ์„ ํ•œ๋‹ค๋Š” ๊ฒŒ ๋ฌด์Šจ๋ง์ผ๊นŒ์š”?

systemd๋Š” /sys/fs/cgroup ์•„๋ž˜์— cgroupfs๋“ค์„ ๋งˆ์šดํŠธํ•ฉ๋‹ˆ๋‹ค.

๊ทธ๋ž˜์„œ mount ๋ช…๋ น์–ด๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ๋งˆ์šดํŠธ๋˜์–ด ์žˆ๋Š” cgroupfs๋“ค์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

 

# mount 
...
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
...

 

 

์œ„์— mount๋œ ํŒŒ์ผ๋“ค์„ ํ™•์ธํ•ด๋ณด๋ฉด ์•Œ ์ˆ˜ ์žˆ๋“ฏ์ด, cgroup์€ ์ œ์–ดํ•˜๋Š” ๋ฆฌ์†Œ์Šค๋ฅผ ํƒ€์ž…๋ณ„๋กœ ๊ด€๋ฆฌํ•ฉ๋‹ˆ๋‹ค.

๊ฐ๊ฐ์˜ ํƒ€์ž…์— ๋งž์ถฐ ๋”ฐ๋กœ mountํ•ด์„œ ๊ด€๋ฆฌํ•˜๋Š” ๊ฒƒ์ด์ฃ .

 

์ด๋ ‡๊ฒŒ ํŠน์ˆ˜ํ•œ ํ˜•ํƒœ์˜ ํŒŒ์ผ์‹œ์Šคํ…œ์œผ๋กœ ๊ด€๋ฆฌ๋˜์–ด์ง€๊ธฐ ๋•Œ๋ฌธ์—, 

cgroupfs์œผ๋กœ cgroup์„ ์ œ์–ดํ• ๋•Œ๋Š” mkdir, rmdir, echo์™€ ๊ฐ™์€ ๋ช…๋ น์–ด๋“ค์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

 

์˜ˆ๋ฅผ ๋“ค์–ด, cgroup์„ ์ƒ์„ฑํ•˜๋Š” ๋ฐฉ๋ฒ•์€ cgroupfs์— ๋””๋ ‰ํ„ฐ๋ฆฌ๋ฅผ ์ƒ์„ฑํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.

cgroupfs ๋‚ด ๋””๋ ‰ํ„ฐ๋ฆฌ๋Š” ํ•˜๋‚˜์˜ cgroup์ด๋ฉฐ, cgroup์€ ๋””๋ ‰ํ„ฐ๋ฆฌ ๊ตฌ์กฐ์ฒ˜๋Ÿผ Tree ํ˜•ํƒœ๋ฅผ ๊ฐ–์Šต๋‹ˆ๋‹ค.

 

 

 

 

์ด๋ ‡๊ฒŒ Docker Engine์„ ๋‹ค๋ฃจ๋Š” ๋‘ ๋ฒˆ์งธ ํฌ์ŠคํŒ…์„ ๋งˆ์น˜๊ฒ ์Šต๋‹ˆ๋‹ค.

์ปจํ…Œ์ด๋„ˆ ๊ธฐ์ˆ ์€ ์ด๋ ‡๊ฒŒ namespace, cgroup ๋ฅผ ํ†ตํ•ด ๋…๋ฆฝ์ ์œผ๋กœ ๊ฒฉ๋ฆฌ๋  ์ˆ˜ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค.

๊ถ๊ธˆ์ฆ์ด ํ’€๋ฆฌ๋ฉด์„œ ๋ญ”๊ฐ€ ๊ฐˆ์ฆ์ด ํ•ด๊ฒฐ๋œ ๊ฒƒ ๊ฐ™์•„์š” ๐Ÿ‘Š๐Ÿป

๋‹ค์Œ ํฌ์ŠคํŒ…์€ network, volume์— ๋Œ€ํ•œ ๊นŠ์€ ๋‚ด์šฉ์„ ๋‹ค๋ฃจ๊ฒ ์Šต๋‹ˆ๋‹ค.

 

๋ถ€์กฑํ•œ ๋‚ด์šฉ์ด๋‚˜, ์ฒจ์–ธ, ํ”ผ๋“œ๋ฐฑ ๋‚ด์šฉ์ด ์žˆ๋‹ค๋ฉด ๋Œ“๊ธ€๋กœ ๋‚จ๊ฒจ์ฃผ์‹œ๋ฉด ์ •๋ง์ •๋ง ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

 

 

 

์ฐธ๊ณ  ์ž๋ฃŒ

docking a docker container part 2

LXC vs Libcontainer

UTS namespace

kakao - cgroup driver