How to Build K8S
tags: container
How to Build K8S
本文编写时，最新release为v1.22.4, 因此下文分析的代码均为v1.22.4分支的代码。1. build
根据官方文档，一步一步来即可
1.下载代码
git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes
git checkout v1.22.4
2.修改代码
sed -i "s/%s %s/%s %s modified by st0n3/g" staging/src/k8s.io/component-base/version/verflag/verflag.go
3.build: 默认会为所有模块编译出所有架构的代码，我们也可以指定编译的模块、架构。
./build/run.sh make kubelet KUBE_BUILD_PLATFORMS=linux/amd64
4.binary: 编译好的二进制文件位于./_output/dockerized/bin目录
# ./_output/dockerized/bin/linux/amd64/kubelet --version
Kubernetes v1.22.4 modified by st0n3
2. advance
2.1 编译过程分析
上面我们执行的 run.sh 主要执行了3个步骤(编译构建环境、执行编译命令、复制编译好的文件)，这些步骤都是定义在common.sh中的函数。其中kube::build::run_build_command函数，直接接受外部输入，所以当我们执行run.sh make时，等同于执行kube::build::run_build_command make
https://github.com/kubernetes/kubernetes/blob/v1.22.4/build/run.sh#L39
source "$KUBE_ROOT/build/common.sh"
...
kube::build::build_image
...
kube::build::run_build_command "$@"
...
kube::build::copy_output
分析kube::build::run_build_command()函数，发现传入的cmd参数，由docker run命令传入上面构建好的容器中，例如我们执行run.sh make命令，将在容器中执行make命令。因此，如果我们可以进入这个容器中，则可以自定义构建环境。
https://github.com/kubernetes/kubernetes/blob/v1.22.4/build/common.sh#L470-L558
function kube::build::run_build_command() {
kube::log::status "Running build command..."
kube::build::run_build_command_ex "${KUBE_BUILD_CONTAINER_NAME}" -- "$@"
}
function kube::build::run_build_command_ex() {
...
local -a cmd=()
until [ -z "${1-}" ] ; do
cmd+=("$1")
shift
done
docker_run_opts+=(
--env "KUBE_FASTBUILD=${KUBE_FASTBUILD:-false}"
--env "KUBE_BUILDER_OS=${OSTYPE:-notdetected}"
--env "KUBE_VERBOSE=${KUBE_VERBOSE}"
--env "KUBE_BUILD_WITH_COVERAGE=${KUBE_BUILD_WITH_COVERAGE:-}"
--env "KUBE_BUILD_PLATFORMS=${KUBE_BUILD_PLATFORMS:-}"
--env "GOFLAGS=${GOFLAGS:-}"
--env "GOGCFLAGS=${GOGCFLAGS:-}"
--env "SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-}"
)
...
local -ra docker_cmd=(
"${DOCKER[@]}" run "${docker_run_opts[@]}" "${KUBE_BUILD_IMAGE}")
...
# Clean up container from any previous run
kube::build::destroy_container "${container_name}"
"${docker_cmd[@]}" "${cmd[@]}"
if [[ "${detach}" == false ]]; then
kube::build::destroy_container "${container_name}"
fi
}
文档中已经说明，执行./build/shell.sh可以进入到容器内。
2.2 进入编译环境，手动编译
root@host:~/kubernetes# ./build/shell.sh
+++ [1124 15:35:18] Verifying Prerequisites....
+++ [1124 15:35:19] Building Docker image kube-build:build-5fe81ecaa8-5-v1.22.0-go1.16.10-buster.0
+++ [1124 15:35:21] Creating data container kube-build-data-5fe81ecaa8-5-v1.22.0-go1.16.10-buster.0
+++ [1124 15:35:37] Syncing sources to container
+++ [1124 15:35:42] Output from this container will NOT be rsynced out upon completion. Set KUBE_RUN_COPY_OUTPUT=y to enable.
+++ [1124 15:35:42] Running build command...
root@host:~# sed -i "s/%s %s/%s %s modified by st0n3/g" staging/src/k8s.io/component-base/version/verflag/verflag.go
root@host:~# make kubelet KUBE_BUILD_PLATFORMS=linux/amd64
Go version: go version go1.16.10 linux/amd64
+++ [1124 15:36:20] Building go targets for linux/amd64:
...too many logs...
root@host:~# ./_output/bin/kubelet --version
Kubernetes v1.22.4 modified by st0n3
2.3 修改编译命令
根据k8s makefile 源码分析，或在hack/lib/golang.sh中加上 set -ex再重新执行上述命令，可以发现具体的编译命令为
+ go install -gcflags ' -trimpath=/go/src/k8s.io/kubernetes' -asmflags -trimpath=/go/src/k8s.io/kubernetes -ldflags '-s -w -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.buildDate=2021-11-24T02:38:56Z'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.buildDate=2021-11-24T02:38:56Z'\'' -X '\''k8s.io/client-go/pkg/version.buildDate=2021-11-24T02:38:56Z'\'' -X '\''k8s.io/component-base/version.buildDate=2021-11-24T02:38:56Z'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.gitCommit=f572e4d5b4a1496356eccf29c34adac02280beeb'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.gitCommit=f572e4d5b4a1496356eccf29c34adac02280beeb'\'' -X '\''k8s.io/client-go/pkg/version.gitCommit=f572e4d5b4a1496356eccf29c34adac02280beeb'\'' -X '\''k8s.io/component-base/version.gitCommit=f572e4d5b4a1496356eccf29c34adac02280beeb'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.gitTreeState=dirty'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.gitTreeState=dirty'\'' -X '\''k8s.io/client-go/pkg/version.gitTreeState=dirty'\'' -X '\''k8s.io/component-base/version.gitTreeState=dirty'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.gitVersion=v1.23.0-beta.0.82+f572e4d5b4a149-dirty'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.gitVersion=v1.23.0-beta.0.82+f572e4d5b4a149-dirty'\'' -X '\''k8s.io/client-go/pkg/version.gitVersion=v1.23.0-beta.0.82+f572e4d5b4a149-dirty'\'' -X '\''k8s.io/component-base/version.gitVersion=v1.23.0-beta.0.82+f572e4d5b4a149-dirty'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.gitMajor=1'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.gitMajor=1'\'' -X '\''k8s.io/client-go/pkg/version.gitMajor=1'\'' -X '\''k8s.io/component-base/version.gitMajor=1'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/client-go/pkg/version.gitMinor=23+'\'' -X '\''k8s.io/kubernetes/vendor/k8s.io/component-base/version.gitMinor=23+'\'' -X '\''k8s.io/client-go/pkg/version.gitMinor=23+'\'' -X '\''k8s.io/component-base/version.gitMinor=23+'\''' -tags selinux,notest, k8s.io/kubernetes/cmd/kubelet
因此可以通过修改hack/lib/golang.sh中go install相关的参数来修改编译命令
https://github.com/kubernetes/kubernetes/blob/master/hack/lib/golang.sh#L685
go install "${build_args[@]}" "$@"
3. troubleshooting
3.1 GFW
TODO