>>> kops: Building testing/kops 1.25.1-r0 (using abuild 3.10.0_rc1-r2) started Tue, 18 Oct 2022 05:12:00 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r2) (2/3) Installing go (1.19.2-r0) (3/3) Installing .makedepends-kops (20221018.051201) Executing busybox-1.35.0-r27.trigger OK: 800 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.25.1.tar.gz::https://github.com/kubernetes/kops/archive/v1.25.1.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 352k 0 352k 0 0 508k 0 --:--:-- --:--:-- --:--:-- 508k 100 10.4M 0 10.4M 0 0 6313k 0 --:--:-- 0:00:01 --:--:-- 10.0M 100 15.0M 0 15.0M 0 0 5712k 0 --:--:-- 0:00:02 --:--:-- 7512k 100 20.4M 0 20.4M 0 0 5684k 0 --:--:-- 0:00:03 --:--:-- 6879k 100 24.7M 0 24.7M 0 0 5378k 0 --:--:-- 0:00:04 --:--:-- 6220k 100 29.4M 0 29.4M 0 0 5284k 0 --:--:-- 0:00:05 --:--:-- 5946k 100 30.8M 0 30.8M 0 0 5199k 0 --:--:-- 0:00:06 --:--:-- 4770k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.1.tar.gz >>> kops: Checking sha512sums... kops-1.25.1.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.1.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.103 go: downloading k8s.io/apimachinery v0.25.2 go: downloading k8s.io/cli-runtime v0.25.2 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading github.com/cert-manager/cert-manager v1.9.1 go: downloading k8s.io/client-go v0.25.2 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/api v0.25.2 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/spf13/cobra v1.5.0 go: downloading github.com/spf13/viper v1.13.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading golang.org/x/crypto v0.0.0-20220919173607-35f4265a4bc0 go: downloading github.com/stretchr/testify v1.8.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading github.com/hashicorp/vault/api v1.8.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading golang.org/x/oauth2 v0.0.0-20220909003341-f21342109be1 go: downloading google.golang.org/api v0.97.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/utils v0.0.0-20220922104903-7796b5f52b7e go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/controller-runtime v0.13.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading k8s.io/component-base v0.25.2 go: downloading github.com/prometheus/client_golang v1.13.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.11.0 go: downloading helm.sh/helm/v3 v3.10.0 go: downloading k8s.io/kubectl v0.25.2 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/digitalocean/godo v1.85.0 go: downloading cloud.google.com/go/compute v1.10.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.2 go: downloading github.com/hetznercloud/hcloud-go v1.35.3 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.2 go: downloading k8s.io/mount-utils v0.25.2 go: downloading golang.org/x/sys v0.0.0-20220919091848-fb04ddd9f9c8 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading golang.org/x/net v0.0.0-20220921203646-d300de134e69 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading k8s.io/cloud-provider-aws v1.25.0 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/spotinst/spotinst-sdk-go v1.129.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/sync v0.0.0-20220907140024-f12130a52804 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.14.0 go: downloading github.com/zclconf/go-cty v1.11.0 go: downloading k8s.io/gengo v0.0.0-20220913193501-391367153a38 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.20 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading golang.org/x/time v0.0.0-20220609170525-579cf78fd858 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.3.7 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading google.golang.org/appengine v1.6.7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading google.golang.org/grpc v1.48.0 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading k8s.io/cloud-provider v0.25.2 go: downloading k8s.io/csi-translation-lib v0.25.2 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/docker/cli v20.10.17+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.0 go: downloading k8s.io/apiextensions-apiserver v0.25.0 go: downloading sigs.k8s.io/gateway-api v0.4.3 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading google.golang.org/genproto v0.0.0-20220915135415-7fd63a7952de go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/onsi/gomega v1.20.1 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/stretchr/objx v0.4.0 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/klauspost/compress v1.15.8 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/onsi/ginkgo/v2 v2.1.6 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/docker/docker v20.10.17+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.0 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/containerd/containerd v1.6.6 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.1/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.1/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I1018 05:14:11.881168 28628 addon.go:233] addon test wants to update all nodes I1018 05:14:11.883601 28628 addon.go:233] addon test wants to update worker nodes I1018 05:14:11.884051 28628 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I1018 05:14:11.884588 28628 addon.go:266] installing PKI for "test" I1018 05:14:13.562686 28628 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (2.07s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 2.186s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.081s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I1018 05:14:28.202299 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.202540 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.202595 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.202644 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.202699 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.202790 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.202808 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.202843 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.202869 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.202941 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.202976 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.308820 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.309060 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.309517 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.323681 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.323864 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.323907 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.323945 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.323982 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.324026 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.324048 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.324088 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.324109 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.324189 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.324221 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.375972 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.376124 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.376420 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.380967 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.381096 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.381147 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.381171 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.381218 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.381242 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.381255 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.381277 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.381292 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.381346 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.381376 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.428042 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.428198 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.428485 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.433507 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.433633 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.433667 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.433690 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.433714 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.433739 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.433753 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.433772 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.433785 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.433845 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.433867 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.509305 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.509479 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.509781 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.516118 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.516249 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.516287 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.516322 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.516353 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.516380 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.516396 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.516420 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.516437 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.516493 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.516516 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.540219 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.540399 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.540681 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.546433 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.546595 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.546630 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.546658 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.546689 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.546715 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.546732 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.546755 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.546772 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.546832 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.546858 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.666221 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.666482 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.666935 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.675845 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.676088 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.676142 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.676194 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.676247 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.676291 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.676327 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.676369 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.676404 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.676548 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.676598 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.737225 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.737488 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.737895 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.744085 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.744246 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.744289 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.744320 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.744352 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.744389 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.744406 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.744428 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.744448 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.744506 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.744532 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.783570 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.783749 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.784038 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.59s) === RUN TestCreateClusterHetzner I1018 05:14:28.791345 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.791478 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.791514 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.791539 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.791567 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.791638 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.791665 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.791687 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.791703 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.791770 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.791791 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1018 05:14:28.821720 33299 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1018 05:14:28.821768 33299 new_cluster.go:1279] Cloud Provider ID = hetzner I1018 05:14:28.827399 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.827501 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.827533 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.827556 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.827583 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.827607 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.827623 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.827642 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.827657 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.827712 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.827733 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.842996 33299 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.06s) === RUN TestCreateClusterOverride I1018 05:14:28.849348 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.849505 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.849538 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.849562 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.849589 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.849637 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.849651 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.849670 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.849684 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.849741 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.849762 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.900717 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.901316 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.901632 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterComplex I1018 05:14:28.907679 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.907843 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.907882 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.907913 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.907946 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.908001 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.908020 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.908048 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.908067 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.908130 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.908157 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.976019 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:28.976198 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:28.976496 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.07s) === RUN TestCreateClusterHA I1018 05:14:28.982292 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.982455 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.982493 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.982523 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.982556 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.982622 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.982643 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.982674 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.982694 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.982759 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:28.982784 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.047034 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.047377 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.047394 33299 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1018 05:14:29.047404 33299 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1018 05:14:29.047734 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:29.056363 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.056510 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.056549 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.056583 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.056619 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.056647 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.056669 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.056695 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.056713 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.056778 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.056804 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.096153 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.096506 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.096524 33299 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1018 05:14:29.096534 33299 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1018 05:14:29.096880 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.13s) === RUN TestCreateClusterHAGCE I1018 05:14:29.108368 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.108546 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.108585 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.108618 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.108653 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.108709 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.108727 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.108749 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.108768 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.108834 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.108861 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.150114 33299 new_cluster.go:1279] Cloud Provider ID = gce I1018 05:14:29.150262 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.05s) === RUN TestCreateClusterGCE I1018 05:14:29.160577 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.160750 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.160787 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.160823 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.160857 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.160915 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.160932 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.160955 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.160973 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.161043 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.161069 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.200780 33299 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1018 05:14:29.200847 33299 new_cluster.go:1279] Cloud Provider ID = gce I1018 05:14:29.200923 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.05s) === RUN TestCreateClusterHASharedZone I1018 05:14:29.207187 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.207387 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.207439 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.207487 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.207539 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.207619 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.207648 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.207687 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.207714 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.207802 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.207845 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1018 05:14:29.276923 33299 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1018 05:14:29.277089 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.277583 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.278160 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.08s) === RUN TestCreateClusterHASharedZones I1018 05:14:29.290732 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.291026 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.291115 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.291203 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.291281 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.291449 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.291512 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.291582 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.291653 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.291789 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.291864 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1018 05:14:29.335470 33299 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1018 05:14:29.335534 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.335922 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.335940 33299 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1018 05:14:29.336311 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.06s) === RUN TestCreateClusterPrivate I1018 05:14:29.347893 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.348079 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.348116 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.348151 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.348186 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.348240 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.348259 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.348301 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.348319 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.348388 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.348416 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.385728 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.385952 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.385968 33299 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1018 05:14:29.386317 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:29.386359 33299 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I1018 05:14:29.393183 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.393350 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.393386 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.393416 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.393448 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.393501 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.393519 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.393541 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.393558 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.393616 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.393653 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.436099 33299 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1018 05:14:29.436176 33299 new_cluster.go:1279] Cloud Provider ID = gce I1018 05:14:29.436262 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.05s) === RUN TestCreateClusterWithNGWSpecified I1018 05:14:29.444812 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.445075 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.445130 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.445176 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.445232 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.445316 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.445349 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.445391 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.445422 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.445514 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.445558 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.509384 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.509717 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.509742 33299 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1018 05:14:29.510215 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:29.510260 33299 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.08s) === RUN TestCreateClusterWithINGWSpecified I1018 05:14:29.520073 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.520290 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.520334 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.520376 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.520421 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.520495 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.520526 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.520562 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.520589 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.520670 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.520710 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.581041 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.581257 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.581275 33299 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1018 05:14:29.581576 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:29.581626 33299 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.07s) === RUN TestCreateClusterSharedVPC I1018 05:14:29.587891 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.588058 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.588090 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.588119 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.588151 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.588204 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.588222 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.588245 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.588262 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.588338 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.588364 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.651811 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.651967 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.652021 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.652052 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.652067 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.652108 33299 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I1018 05:14:29.652398 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.07s) === RUN TestCreateClusterSharedSubnets I1018 05:14:29.657702 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.657844 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.657874 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.657901 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.657931 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.657983 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.658002 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.658025 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.658042 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.658097 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.658121 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.696913 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.696961 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.697030 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.697164 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.697187 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.697210 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.697226 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.697524 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.05s) === RUN TestCreateClusterSharedSubnetsVpcLookup I1018 05:14:29.702786 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.702937 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.702964 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.702989 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.703016 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.703067 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.703083 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.703106 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.703123 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.703178 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.703203 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.764634 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:29.764682 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.764711 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.764778 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.764950 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.764976 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.765007 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.765023 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.765338 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.07s) === RUN TestCreateClusterPrivateSharedSubnets I1018 05:14:29.772794 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.772960 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.772994 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.773024 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.773056 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.773119 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.773139 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.773162 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.773180 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.773275 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.773303 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.804890 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.804955 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.805180 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.805203 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.805234 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.805374 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.805396 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.805420 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:14:29.805436 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:29.805760 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.04s) === RUN TestCreateClusterIPv6 I1018 05:14:29.812022 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.812229 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.812268 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.812298 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.812332 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.812387 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.812405 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.812428 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.812446 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.812511 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.812537 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.867990 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.868204 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.868658 33299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.06s) === RUN TestCreateClusterDifferentAMIs I1018 05:14:29.876223 33299 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1018 05:14:29.876244 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.876416 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.876461 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.876494 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.876536 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.876602 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.876638 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.876670 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.876692 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.876768 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.876800 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.919860 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:29.919963 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:29.919982 33299 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1018 05:14:29.920354 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:29.931234 33299 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.06s) === RUN TestCreateClusterKarpenter I1018 05:14:29.931329 33299 featureflag.go:160] FeatureFlag "Karpenter"=true I1018 05:14:29.931340 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.931500 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.931533 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.931562 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.931592 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.931656 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.931672 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.931694 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.931710 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.931774 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:29.931799 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1018 05:14:30.048898 33299 new_cluster.go:1279] Cloud Provider ID = aws I1018 05:14:30.049157 33299 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1018 05:14:30.049180 33299 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1018 05:14:30.049195 33299 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1018 05:14:30.049680 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:30.058660 33299 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.13s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I1018 05:14:30.058984 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.059147 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.059185 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.059260 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.059298 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.059333 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.059358 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.059388 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.059413 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.065226 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I1018 05:14:30.066318 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.066454 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.066480 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.066506 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.066533 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.066573 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.066589 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.066610 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.066623 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.082591 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:30.085608 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:30.085702 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:32.655434 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:35.859196 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:14:35.860229 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:35.862219 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:35.862240 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:35.862322 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:35.862647 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:35.862704 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:35.862772 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:35.871428 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:35.872099 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:35.872500 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:35.873511 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:35.875266 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:35.875416 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:35.875957 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:35.876242 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:35.918922 33299 target.go:112] Terraform output is in /tmp/test2673067524/out I1018 05:14:35.922786 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:35.923093 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:35.923170 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:35.923248 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:35.923337 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:35.923408 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:35.923463 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:35.923531 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:35.923580 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:35.949726 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:35.952700 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:35.952801 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:35.995496 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:36.097568 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:36.098233 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:36.098244 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:36.098357 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:36.098377 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:36.098387 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:36.098379 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:36.098457 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:36.098451 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } W1018 05:14:36.098535 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:36.100480 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:36.104709 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:36.105156 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.106956 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.110556 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.111894 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.112037 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:36.112365 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:36.112541 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:36.117054 33299 target.go:168] Cloudformation output is in /tmp/test2809309298/out --- PASS: TestMinimal (6.06s) === RUN TestMinimal_v1_23 I1018 05:14:36.127589 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:36.127813 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:36.127867 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:36.127956 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:36.128013 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:36.128059 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:36.128091 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:36.128131 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:36.128160 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:36.153435 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1018 05:14:36.156424 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.156505 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.679226 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:36.820874 33299 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1018 05:14:36.821867 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:36.822418 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:36.822440 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:36.822523 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:36.823269 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:36.823711 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:36.825926 33299 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1018 05:14:36.830941 33299 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1018 05:14:36.831153 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.832496 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.834684 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.835907 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.836025 33299 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1018 05:14:36.836318 33299 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1018 05:14:36.836458 33299 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1018 05:14:36.869886 33299 target.go:112] Terraform output is in /tmp/test4160093717/out I1018 05:14:36.872171 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:36.872301 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:36.872341 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:36.872374 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:36.872409 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:36.872437 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:36.872453 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:36.872477 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:36.872495 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:36.885950 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:36.887677 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:36.887735 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:36.925111 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:37.012167 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:37.012686 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:37.013067 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:37.013101 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:37.013154 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:37.013118 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:37.013234 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:37.014268 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:37.014300 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:37.014405 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1018 05:14:37.018084 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:37.021654 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:37.022046 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.022207 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.023125 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.023395 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.023478 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:37.023670 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:37.023786 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:37.026398 33299 target.go:168] Cloudformation output is in /tmp/test211465075/out --- PASS: TestMinimal_v1_23 (0.91s) === RUN TestMinimal_v1_24 I1018 05:14:37.036213 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:37.036389 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:37.036431 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:37.036505 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:37.036555 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:37.036596 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:37.036624 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:37.036662 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:37.036689 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:37.063977 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:37.067327 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.067431 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.598300 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:37.734912 33299 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1018 05:14:37.735324 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:37.735371 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:37.735507 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:37.735542 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:37.735571 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:37.735856 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:37.737357 33299 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1018 05:14:37.741819 33299 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1018 05:14:37.742415 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.742815 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.745228 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.746981 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.747113 33299 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1018 05:14:37.747427 33299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1018 05:14:37.747616 33299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1018 05:14:37.798936 33299 target.go:112] Terraform output is in /tmp/test1901904565/out I1018 05:14:37.801422 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:37.801544 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:37.801584 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:37.801628 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:37.801665 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:37.801693 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:37.801711 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:37.801735 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:37.801753 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:37.815216 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:37.816950 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.817008 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:37.854345 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:37.937916 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:37.938222 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:37.938420 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:37.938596 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:37.938621 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:37.938728 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:37.938646 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:37.938768 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:37.938744 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:37.938833 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:37.940742 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:37.946629 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:37.946901 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.947002 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.947700 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.949727 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.949872 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:37.950149 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:37.950471 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:37.953840 33299 target.go:168] Cloudformation output is in /tmp/test3363300515/out --- PASS: TestMinimal_v1_24 (0.93s) === RUN TestMinimal_v1_25 I1018 05:14:37.963636 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:37.963848 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:37.963899 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:37.963987 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:37.964038 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:37.964084 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:37.964113 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:37.964150 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:37.964178 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:37.989966 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:37.993235 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:37.993324 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.536716 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:38.652992 33299 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1018 05:14:38.653354 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:38.654085 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:38.654212 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:38.654143 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:38.654380 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:38.654284 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:38.657677 33299 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1018 05:14:38.663603 33299 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1018 05:14:38.663773 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.664326 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.664606 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.667570 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.667692 33299 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1018 05:14:38.668021 33299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1018 05:14:38.668284 33299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1018 05:14:38.720031 33299 target.go:112] Terraform output is in /tmp/test3662865845/out I1018 05:14:38.722757 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:38.722910 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:38.722961 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:38.723008 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:38.723062 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:38.723101 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:38.723135 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:38.723175 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:38.723202 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:38.740781 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:38.743373 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.743448 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:38.782129 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:38.879521 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:38.879858 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:38.880589 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:38.880600 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:38.880625 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:38.880715 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:38.880759 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:38.880782 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:38.880760 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:38.880870 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:38.882673 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:38.892355 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:38.892835 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.892956 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.893904 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.895589 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.895735 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:38.896013 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:38.896208 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:38.899469 33299 target.go:168] Cloudformation output is in /tmp/test2421504465/out --- PASS: TestMinimal_v1_25 (0.95s) === RUN TestMinimal_v1_26 I1018 05:14:38.909245 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:38.909459 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:38.909511 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:38.909556 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:38.909605 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:38.909688 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:38.909717 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:38.909755 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:38.909783 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:38.939345 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:38.943248 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:38.943353 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.485276 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:39.660827 33299 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1018 05:14:39.661690 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:39.661730 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:39.661899 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:39.661939 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:39.662037 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:39.662003 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:39.664322 33299 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1018 05:14:39.669665 33299 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1018 05:14:39.670054 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.670287 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.671344 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.672962 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.673083 33299 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1018 05:14:39.673435 33299 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1018 05:14:39.673620 33299 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1018 05:14:39.727328 33299 target.go:112] Terraform output is in /tmp/test3654338856/out I1018 05:14:39.732012 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:39.732229 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:39.732290 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:39.732343 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:39.732399 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:39.732444 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:39.732473 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:39.732510 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:39.732538 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:39.760247 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:39.763763 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.763885 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:39.803849 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:39.906194 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:14:39.906722 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:39.906768 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:39.906757 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:39.906859 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:39.907112 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:39.907246 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:39.907270 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:39.907287 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:39.907399 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:39.909781 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:39.914906 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:39.915459 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.915748 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.916860 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.917777 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:39.917924 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:39.918202 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:39.918410 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:39.921265 33299 target.go:168] Cloudformation output is in /tmp/test176788076/out --- PASS: TestMinimal_v1_26 (1.02s) === RUN TestHetzner I1018 05:14:39.930744 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:40.634019 33299 executor.go:111] Tasks: 0 done / 50 total; 39 can run I1018 05:14:40.640772 33299 executor.go:111] Tasks: 39 done / 50 total; 5 can run I1018 05:14:40.642182 33299 executor.go:111] Tasks: 44 done / 50 total; 2 can run I1018 05:14:40.647240 33299 executor.go:111] Tasks: 46 done / 50 total; 4 can run I1018 05:14:40.650144 33299 executor.go:111] Tasks: 50 done / 50 total; 0 can run I1018 05:14:40.670083 33299 target.go:112] Terraform output is in /tmp/test633291986/out --- PASS: TestHetzner (0.74s) === RUN TestNvidia I1018 05:14:40.673762 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:40.674008 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:40.674068 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:40.674178 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:40.674235 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:40.674283 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:40.674314 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:40.674354 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:40.674386 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:40.698769 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:40.701897 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.702000 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.739821 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:40.846041 33299 executor.go:111] Tasks: 0 done / 88 total; 44 can run I1018 05:14:40.846517 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:40.846573 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:40.846717 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:40.847996 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:40.848102 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:40.848540 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:40.853563 33299 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1018 05:14:40.859872 33299 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1018 05:14:40.860159 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.860467 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.861387 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.863243 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.863362 33299 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1018 05:14:40.863654 33299 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1018 05:14:40.863841 33299 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1018 05:14:40.917002 33299 target.go:112] Terraform output is in /tmp/test141115861/out I1018 05:14:40.921048 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:40.921262 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:40.921323 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:40.921376 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:40.921432 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:40.921528 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:40.921561 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:40.921621 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:40.921657 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:40.950891 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:40.953991 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.954091 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:40.994829 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:41.103916 33299 executor.go:111] Tasks: 0 done / 88 total; 44 can run W1018 05:14:41.104472 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:41.104527 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:41.104645 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:41.104663 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:41.104750 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:41.107550 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:41.107614 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:41.107716 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:41.108229 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:41.111984 33299 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1018 05:14:41.117634 33299 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1018 05:14:41.118068 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.118212 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.119203 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.120710 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.120945 33299 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1018 05:14:41.121224 33299 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1018 05:14:41.121407 33299 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1018 05:14:41.124382 33299 target.go:168] Cloudformation output is in /tmp/test557905144/out --- PASS: TestNvidia (0.46s) === RUN TestMinimalGossip I1018 05:14:41.134147 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:41.134364 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:41.134418 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:41.134462 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:41.134519 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:41.134599 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:41.134628 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:41.134668 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:41.134697 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:41.166234 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.166332 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.310918 33299 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1018 05:14:41.311216 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:41.311529 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:41.314500 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:41.318706 33299 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1018 05:14:41.324307 33299 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1018 05:14:41.324649 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.324903 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.325978 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.327354 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.327474 33299 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1018 05:14:41.327771 33299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1018 05:14:41.328021 33299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1018 05:14:41.380049 33299 target.go:112] Terraform output is in /tmp/test2831024385/out --- PASS: TestMinimalGossip (0.25s) === RUN TestMinimalGossipIRSA I1018 05:14:41.384221 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:41.384450 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:41.384506 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:41.384555 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:41.384608 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:41.384703 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:41.384734 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:41.384776 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:41.384806 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:41.412507 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.412606 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.566456 33299 executor.go:111] Tasks: 0 done / 89 total; 44 can run W1018 05:14:41.566942 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:41.567101 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:41.567564 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:41.569914 33299 executor.go:111] Tasks: 44 done / 89 total; 20 can run I1018 05:14:41.575173 33299 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1018 05:14:41.575430 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.575819 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.576986 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.577978 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:41.578103 33299 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1018 05:14:41.578465 33299 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1018 05:14:41.578734 33299 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1018 05:14:41.631980 33299 target.go:112] Terraform output is in /tmp/test569649193/out --- PASS: TestMinimalGossipIRSA (0.25s) === RUN TestMinimalGCE I1018 05:14:41.636076 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:41.637928 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1018 05:14:42.107727 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:42.107836 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:42.107950 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1018 05:14:42.107991 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1018 05:14:42.108017 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:14:42.108051 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:14:42.108132 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:14:42.109668 33299 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1018 05:14:42.116672 33299 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1018 05:14:42.122335 33299 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1018 05:14:42.125455 33299 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1018 05:14:42.125974 33299 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1018 05:14:42.154122 33299 target.go:112] Terraform output is in /tmp/test142707870/out --- PASS: TestMinimalGCE (0.52s) === RUN TestMinimalGCEPrivate I1018 05:14:42.158251 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:42.160091 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:14:42.594645 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:42.594734 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:42.594867 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W1018 05:14:42.594905 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W1018 05:14:42.594956 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1018 05:14:42.595006 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1018 05:14:42.595080 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:14:42.596702 33299 executor.go:111] Tasks: 0 done / 66 total; 40 can run I1018 05:14:42.600902 33299 executor.go:111] Tasks: 40 done / 66 total; 18 can run I1018 05:14:42.608790 33299 executor.go:111] Tasks: 58 done / 66 total; 4 can run I1018 05:14:42.612762 33299 executor.go:111] Tasks: 62 done / 66 total; 3 can run I1018 05:14:42.614180 33299 executor.go:111] Tasks: 65 done / 66 total; 1 can run I1018 05:14:42.614349 33299 executor.go:111] Tasks: 66 done / 66 total; 0 can run I1018 05:14:42.644706 33299 target.go:112] Terraform output is in /tmp/test1322573568/out --- PASS: TestMinimalGCEPrivate (0.49s) === RUN TestMinimalGCEInternalLoadBalancer I1018 05:14:42.648319 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:42.650204 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:14:42.829651 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:42.829730 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:42.829864 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W1018 05:14:42.829902 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W1018 05:14:42.829927 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1018 05:14:42.829963 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1018 05:14:42.830039 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1018 05:14:42.830087 33299 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1018 05:14:42.831873 33299 executor.go:111] Tasks: 0 done / 68 total; 41 can run I1018 05:14:42.835055 33299 executor.go:111] Tasks: 41 done / 68 total; 17 can run W1018 05:14:42.840248 33299 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1018 05:14:42.840296 33299 executor.go:111] Tasks: 58 done / 68 total; 4 can run I1018 05:14:42.842813 33299 executor.go:111] Tasks: 62 done / 68 total; 3 can run I1018 05:14:42.843983 33299 executor.go:111] Tasks: 65 done / 68 total; 2 can run I1018 05:14:42.844314 33299 executor.go:111] Tasks: 67 done / 68 total; 1 can run I1018 05:14:42.844569 33299 executor.go:111] Tasks: 68 done / 68 total; 0 can run I1018 05:14:42.874244 33299 target.go:112] Terraform output is in /tmp/test810835665/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.23s) === RUN TestMinimalGCELongClusterName I1018 05:14:42.877771 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:42.879556 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1018 05:14:43.116872 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:43.116978 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:43.117109 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1018 05:14:43.117146 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1018 05:14:43.117173 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1018 05:14:43.117206 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1018 05:14:43.117279 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:14:43.118982 33299 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1018 05:14:43.126265 33299 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1018 05:14:43.132844 33299 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1018 05:14:43.135753 33299 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1018 05:14:43.135993 33299 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1018 05:14:43.166332 33299 target.go:112] Terraform output is in /tmp/test2173076839/out --- PASS: TestMinimalGCELongClusterName (0.29s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I1018 05:14:43.170077 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:43.172022 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1018 05:14:43.405339 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:43.405432 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:43.405580 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1018 05:14:43.405644 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1018 05:14:43.405673 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1018 05:14:43.405699 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1018 05:14:43.405772 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1018 05:14:43.405823 33299 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1018 05:14:43.407582 33299 executor.go:111] Tasks: 0 done / 67 total; 42 can run I1018 05:14:43.414035 33299 executor.go:111] Tasks: 42 done / 67 total; 16 can run W1018 05:14:43.420340 33299 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1018 05:14:43.420387 33299 executor.go:111] Tasks: 58 done / 67 total; 5 can run I1018 05:14:43.423522 33299 executor.go:111] Tasks: 63 done / 67 total; 2 can run I1018 05:14:43.423797 33299 executor.go:111] Tasks: 65 done / 67 total; 1 can run I1018 05:14:43.423967 33299 executor.go:111] Tasks: 66 done / 67 total; 1 can run I1018 05:14:43.424104 33299 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1018 05:14:43.457300 33299 target.go:112] Terraform output is in /tmp/test4204876243/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.29s) === RUN TestHA I1018 05:14:43.461347 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:43.461583 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:43.461670 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:43.461726 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:43.461780 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:43.461876 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:43.461908 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:43.461951 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:43.462003 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:43.494629 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:43.497776 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.497852 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.497901 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.497948 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.537219 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:43.650421 33299 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1018 05:14:43.650901 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:43.650947 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:43.651064 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:43.651097 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:43.651192 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:43.651313 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:43.651339 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:43.651458 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:43.653779 33299 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1018 05:14:43.662642 33299 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1018 05:14:43.662977 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.663109 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.663356 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.663598 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.664869 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.665780 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.665996 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.666139 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:43.666268 33299 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1018 05:14:43.666728 33299 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1018 05:14:43.667003 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:14:43.748008 33299 target.go:112] Terraform output is in /tmp/test363192379/out --- PASS: TestHA (0.29s) === RUN TestHighAvailabilityGCE I1018 05:14:43.753249 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:43.755360 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:14:43.944510 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:14:43.944625 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:14:43.944736 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1018 05:14:43.944797 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W1018 05:14:43.944824 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W1018 05:14:43.944849 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1018 05:14:43.944916 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1018 05:14:43.945014 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1018 05:14:43.945079 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:14:43.947271 33299 executor.go:111] Tasks: 0 done / 83 total; 48 can run I1018 05:14:43.952628 33299 executor.go:111] Tasks: 48 done / 83 total; 20 can run I1018 05:14:43.963080 33299 executor.go:111] Tasks: 68 done / 83 total; 7 can run I1018 05:14:43.966600 33299 executor.go:111] Tasks: 75 done / 83 total; 5 can run I1018 05:14:43.967932 33299 executor.go:111] Tasks: 80 done / 83 total; 3 can run I1018 05:14:43.968212 33299 executor.go:111] Tasks: 83 done / 83 total; 0 can run I1018 05:14:44.012576 33299 target.go:112] Terraform output is in /tmp/test3897938282/out --- PASS: TestHighAvailabilityGCE (0.26s) === RUN TestComplex I1018 05:14:44.017019 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:44.017226 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:44.017286 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:44.017384 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:44.017438 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:44.017485 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:44.017517 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:44.017560 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:44.017592 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:44.051439 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:44.055262 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.055349 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.592099 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:44.784980 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1018 05:14:44.785349 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:44.785390 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:44.785511 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:44.787559 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:44.788231 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:44.791679 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:14:44.798576 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:14:44.798925 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.799118 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.800562 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.801738 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.801865 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:14:44.802261 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:14:44.802466 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:14:44.876851 33299 target.go:112] Terraform output is in /tmp/test2876878359/out I1018 05:14:44.881200 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:44.881417 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:44.881481 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:44.881534 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:44.881589 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:44.881659 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:44.881691 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:44.881734 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:44.881769 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:44.913879 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:44.917988 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.918089 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:44.959626 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:45.149557 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1018 05:14:45.149873 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:45.152225 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:45.152280 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:45.152389 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:45.152490 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:45.157248 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:14:45.163324 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:14:45.163985 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.164234 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.165535 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.167236 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.167382 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:14:45.167671 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:14:45.167827 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:14:45.172388 33299 target.go:168] Cloudformation output is in /tmp/test3367361551/out I1018 05:14:45.188811 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:45.189065 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:45.189121 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:45.189174 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:45.189230 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:45.189275 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:45.189304 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:45.189340 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:45.189371 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:45.222463 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:45.226794 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.226903 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.266034 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:45.460205 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1018 05:14:45.460530 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:45.461177 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:45.461192 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:45.461246 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:45.461386 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:45.465858 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:14:45.473618 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:14:45.474194 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.474434 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.476007 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.477714 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.477834 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:14:45.478247 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:14:45.478538 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:14:45.553491 33299 target.go:112] Terraform output is in /tmp/test3102424056/out --- PASS: TestComplex (1.54s) === RUN TestCompress I1018 05:14:45.557782 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:45.558019 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:45.558082 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:45.558137 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:45.558193 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:45.558314 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:45.558345 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:45.558383 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:45.558415 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:45.581945 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:45.585182 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.585282 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.625000 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:45.725080 33299 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1018 05:14:45.725330 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:45.725444 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:45.725468 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:45.725576 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:45.726128 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:45.727863 33299 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1018 05:14:45.737709 33299 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1018 05:14:45.738324 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.738585 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.740792 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.742894 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.743096 33299 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1018 05:14:45.743606 33299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1018 05:14:45.743801 33299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1018 05:14:45.793990 33299 target.go:112] Terraform output is in /tmp/test2219208559/out --- PASS: TestCompress (0.24s) === RUN TestExternalPolicies I1018 05:14:45.797855 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:45.798079 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:45.798134 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:45.798186 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:45.798239 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:45.798360 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:45.798392 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:45.798428 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:45.798456 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:45.826016 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:45.832770 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.832913 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.872490 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:45.975919 33299 executor.go:111] Tasks: 0 done / 100 total; 45 can run W1018 05:14:45.976494 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:45.976681 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:45.976706 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:45.976726 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:45.976862 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:45.976940 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:45.978706 33299 executor.go:111] Tasks: 45 done / 100 total; 20 can run I1018 05:14:45.984331 33299 executor.go:111] Tasks: 65 done / 100 total; 30 can run I1018 05:14:45.984803 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.985088 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.985897 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.987540 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:45.987660 33299 executor.go:111] Tasks: 95 done / 100 total; 3 can run I1018 05:14:45.988079 33299 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1018 05:14:45.988349 33299 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1018 05:14:46.034521 33299 target.go:112] Terraform output is in /tmp/test1136275225/out --- PASS: TestExternalPolicies (0.24s) === RUN TestMinimalIPv6 I1018 05:14:46.037832 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:46.038075 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:46.038140 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:46.038250 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:46.038328 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:46.038376 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:46.038411 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:46.038450 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:46.038481 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:46.067271 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:46.071168 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.071270 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.112278 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:46.274124 33299 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1018 05:14:46.275194 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:46.276034 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:46.276103 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:46.276286 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:46.278532 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:46.278606 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:46.280627 33299 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1018 05:14:46.286717 33299 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1018 05:14:46.287326 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.287494 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.288676 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.290225 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.290364 33299 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1018 05:14:46.291568 33299 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1018 05:14:46.292233 33299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1018 05:14:46.357079 33299 target.go:112] Terraform output is in /tmp/test1514547646/out I1018 05:14:46.361250 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:46.361458 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:46.361520 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:46.361571 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:46.361648 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:46.361694 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:46.361724 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:46.361762 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:46.361791 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:46.387195 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:46.390853 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.390949 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.430791 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:46.603524 33299 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1018 05:14:46.603954 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:46.604427 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:46.604463 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:46.604575 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:46.604594 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:46.604612 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:46.604609 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:46.604716 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:14:46.604794 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:46.607183 33299 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1018 05:14:46.612608 33299 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1018 05:14:46.612871 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.613021 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.616489 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.619354 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.619538 33299 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1018 05:14:46.619991 33299 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1018 05:14:46.620290 33299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1018 05:14:46.623852 33299 target.go:168] Cloudformation output is in /tmp/test2214331439/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I1018 05:14:46.636205 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:46.636434 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:46.636488 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:46.636536 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:46.636585 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:46.636665 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:46.636694 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:46.636735 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:46.636767 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:46.665273 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1018 05:14:46.668575 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.668659 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.707636 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:46.861037 33299 executor.go:111] Tasks: 0 done / 124 total; 46 can run I1018 05:14:46.861969 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:46.862088 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:46.862186 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:46.862127 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:46.862254 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:46.863373 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:46.867907 33299 executor.go:111] Tasks: 46 done / 124 total; 23 can run I1018 05:14:46.873711 33299 executor.go:111] Tasks: 69 done / 124 total; 36 can run I1018 05:14:46.874118 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.874352 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.875962 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.877261 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.877400 33299 executor.go:111] Tasks: 105 done / 124 total; 11 can run I1018 05:14:46.878049 33299 executor.go:111] Tasks: 116 done / 124 total; 8 can run I1018 05:14:46.878506 33299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1018 05:14:46.954692 33299 target.go:112] Terraform output is in /tmp/test1784381944/out --- PASS: TestMinimalIPv6Private (0.32s) === RUN TestMinimalIPv6Calico I1018 05:14:46.959428 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:46.959687 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:46.959767 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:46.959902 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:46.959995 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:46.960058 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:46.960122 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:46.960191 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:46.960239 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:46.986768 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:46.990447 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:46.990572 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.032148 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:47.490779 33299 executor.go:111] Tasks: 0 done / 109 total; 47 can run I1018 05:14:47.491112 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:47.491155 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:47.491152 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1018 05:14:47.491273 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:47.491293 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:47.491533 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:47.493335 33299 executor.go:111] Tasks: 47 done / 109 total; 21 can run I1018 05:14:47.498795 33299 executor.go:111] Tasks: 68 done / 109 total; 31 can run I1018 05:14:47.499406 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.499630 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.500971 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.502169 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.502306 33299 executor.go:111] Tasks: 99 done / 109 total; 5 can run I1018 05:14:47.502757 33299 executor.go:111] Tasks: 104 done / 109 total; 5 can run I1018 05:14:47.503119 33299 executor.go:111] Tasks: 109 done / 109 total; 0 can run I1018 05:14:47.567859 33299 target.go:112] Terraform output is in /tmp/test370303539/out --- PASS: TestMinimalIPv6Calico (0.62s) === RUN TestMinimalIPv6Cilium I1018 05:14:47.576424 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:47.576670 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:47.576733 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:47.576839 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:47.576893 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:47.576939 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:47.576969 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:47.577007 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:47.577038 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:47.602744 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:47.606963 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.607060 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.646547 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:47.853830 33299 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1018 05:14:47.854079 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:47.857480 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:47.858200 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:47.858251 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:47.858407 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:47.858521 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:47.861729 33299 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1018 05:14:47.867197 33299 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1018 05:14:47.867672 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.867896 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.868920 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.871052 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.871202 33299 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1018 05:14:47.871663 33299 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1018 05:14:47.872035 33299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1018 05:14:47.937425 33299 target.go:112] Terraform output is in /tmp/test1181533092/out I1018 05:14:47.941866 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:47.942088 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:47.942151 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:47.942205 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:47.942263 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:47.942308 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:47.942337 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:47.942374 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:47.942404 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:47.967979 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:47.971798 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:47.971896 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.011430 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:48.221347 33299 executor.go:111] Tasks: 0 done / 107 total; 46 can run I1018 05:14:48.222911 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:48.223006 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:48.222945 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:48.223054 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:48.224256 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:48.224478 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:48.224608 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:48.224623 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:48.224713 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:14:48.228560 33299 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1018 05:14:48.234435 33299 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1018 05:14:48.234800 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.234927 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.236192 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.237498 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.237658 33299 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1018 05:14:48.238025 33299 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1018 05:14:48.238347 33299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1018 05:14:48.242139 33299 target.go:168] Cloudformation output is in /tmp/test3156508721/out --- PASS: TestMinimalIPv6Cilium (0.68s) === RUN TestMinimalWarmPool I1018 05:14:48.253425 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:48.253646 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:48.253699 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:48.253746 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:48.253801 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:48.253883 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:48.253915 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:48.253954 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:48.253985 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:48.283619 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:48.287150 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.287249 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.326561 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:48.532397 33299 executor.go:111] Tasks: 0 done / 89 total; 45 can run W1018 05:14:48.532898 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:48.533058 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:48.533109 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:48.533240 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:48.533321 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:48.533485 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:48.535424 33299 executor.go:111] Tasks: 45 done / 89 total; 19 can run I1018 05:14:48.540715 33299 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1018 05:14:48.541282 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.541429 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.545878 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.547649 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.547784 33299 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1018 05:14:48.548074 33299 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1018 05:14:48.548324 33299 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1018 05:14:48.597585 33299 target.go:112] Terraform output is in /tmp/test3320772451/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I1018 05:14:48.602462 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:48.602697 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:48.602758 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:48.602868 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:48.602922 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:48.602966 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:48.602996 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:48.603038 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:48.603067 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:48.633942 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:48.637037 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.637134 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.676325 33299 zones.go:93] ListHostedZonesPages { } W1018 05:14:48.755818 33299 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W1018 05:14:48.755883 33299 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W1018 05:14:48.757386 33299 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:48.771360 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:14:48.771887 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:48.772033 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W1018 05:14:48.772075 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:48.772053 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:48.772152 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:48.772177 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:48.772162 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I1018 05:14:48.772264 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:48.772300 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:48.773930 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:48.779167 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:48.779397 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.779720 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.780896 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.782134 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.782269 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:48.782556 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:48.782769 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:48.785771 33299 target.go:168] Cloudformation output is in /tmp/test2163666704/out --- PASS: TestMinimalEtcd (0.19s) === RUN TestMinimalGp3 I1018 05:14:48.795557 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:48.795774 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:48.795827 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:48.795873 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:48.795925 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:48.796005 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:48.796036 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:48.796074 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:48.796104 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:48.825767 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:48.829300 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.829401 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.869255 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:48.973056 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:14:48.973423 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:48.973452 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:48.973540 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:48.973689 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:48.973798 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:48.973829 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:48.976240 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:48.981871 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:48.982300 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.982520 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.983380 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.984863 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:48.985000 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:48.985307 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:48.985557 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:49.038041 33299 target.go:112] Terraform output is in /tmp/test266590219/out I1018 05:14:49.041960 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:49.042215 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:49.042274 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:49.042336 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:49.042403 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:49.042461 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:49.042491 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:49.042538 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:49.042572 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:49.067524 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:49.070797 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.070898 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.110061 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:49.210007 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:14:49.210270 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:49.210297 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:49.210392 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:49.210490 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:49.210720 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:49.210853 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:49.210872 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:49.210959 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:49.210981 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:49.213107 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:49.218717 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:49.219179 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.219430 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.221099 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.225551 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.225717 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:49.226030 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:49.226258 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:49.229391 33299 target.go:168] Cloudformation output is in /tmp/test2282712585/out --- PASS: TestMinimalGp3 (0.44s) === RUN TestMinimalLongClusterName I1018 05:14:49.240391 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:49.240628 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:49.240681 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:49.240730 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:49.240781 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:49.240859 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:49.240889 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:49.240936 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:49.240968 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:49.264264 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:49.267312 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.267405 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:49.306821 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:49.412325 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:49.412593 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:49.412786 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:49.413340 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:49.413484 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:49.413527 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:49.413663 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:49.415029 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:49.420186 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:49.420627 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.420839 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.421750 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.423132 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.423250 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:49.423561 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:49.423838 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:49.482749 33299 target.go:112] Terraform output is in /tmp/test2155500176/out I1018 05:14:49.486601 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:49.486816 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:49.486876 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:49.486930 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:49.486985 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:49.487028 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:49.487089 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:49.487131 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:49.487160 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:49.510988 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:49.514132 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.514228 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:14:49.553565 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:49.653358 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:14:49.653979 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:49.654215 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:49.654205 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:49.654369 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:49.654422 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:49.655657 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:49.655768 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:49.654394 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:49.656519 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" }] }] } I1018 05:14:49.661514 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:14:49.667143 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:14:49.667609 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.667867 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.668790 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.670856 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.671001 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:14:49.671268 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:14:49.671462 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:14:49.675000 33299 target.go:168] Cloudformation output is in /tmp/test4170899916/out --- PASS: TestMinimalLongClusterName (0.44s) === RUN TestExistingIAMCloudformation I1018 05:14:49.685142 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:49.685364 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:49.685416 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:49.685504 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:49.685559 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:49.685602 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:49.685634 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:49.685672 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:49.685701 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:49.714602 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:49.718191 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.718304 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.757927 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:49.860078 33299 executor.go:111] Tasks: 0 done / 77 total; 41 can run W1018 05:14:49.860417 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:49.860685 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:49.860817 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:49.860832 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:49.860951 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:49.860939 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:14:49.861219 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:49.861262 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:49.861429 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:49.864786 33299 executor.go:111] Tasks: 41 done / 77 total; 11 can run I1018 05:14:49.872854 33299 executor.go:111] Tasks: 52 done / 77 total; 21 can run I1018 05:14:49.873324 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.873617 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.874597 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.876049 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.876285 33299 executor.go:111] Tasks: 73 done / 77 total; 2 can run I1018 05:14:49.876733 33299 executor.go:111] Tasks: 75 done / 77 total; 2 can run I1018 05:14:49.877030 33299 executor.go:111] Tasks: 77 done / 77 total; 0 can run I1018 05:14:49.879389 33299 target.go:168] Cloudformation output is in /tmp/test818630614/out --- PASS: TestExistingIAMCloudformation (0.20s) === RUN TestExistingSG I1018 05:14:49.887432 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:49.887659 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:49.887711 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:49.887758 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:49.887814 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:49.887900 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:49.887930 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:49.887967 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:49.887997 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:49.924869 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:49.928544 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.928649 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.928712 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.928772 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:49.968160 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:50.081839 33299 executor.go:111] Tasks: 0 done / 146 total; 53 can run I1018 05:14:50.082406 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:50.082720 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:50.082761 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:50.082816 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:50.082876 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:50.082938 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:50.082963 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:50.083087 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:50.085268 33299 executor.go:111] Tasks: 53 done / 146 total; 26 can run I1018 05:14:50.095701 33299 executor.go:111] Tasks: 79 done / 146 total; 58 can run I1018 05:14:50.096142 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.096374 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.096685 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.096896 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.097388 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.098597 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.098785 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.099378 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.099505 33299 executor.go:111] Tasks: 137 done / 146 total; 5 can run I1018 05:14:50.099951 33299 executor.go:111] Tasks: 142 done / 146 total; 4 can run I1018 05:14:50.100221 33299 executor.go:111] Tasks: 146 done / 146 total; 0 can run I1018 05:14:50.197523 33299 target.go:112] Terraform output is in /tmp/test1354993109/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I1018 05:14:50.202621 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:50.202860 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:50.202925 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:50.202980 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:50.203037 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:50.203133 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:50.203164 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:50.203202 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:50.203233 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:50.232801 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:50.235991 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.236078 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.236140 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.275177 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:50.375931 33299 executor.go:111] Tasks: 0 done / 122 total; 48 can run I1018 05:14:50.376552 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:50.376594 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:50.376698 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:50.376729 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:50.376760 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:50.377099 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:50.378056 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:50.384231 33299 executor.go:111] Tasks: 48 done / 122 total; 29 can run I1018 05:14:50.389728 33299 executor.go:111] Tasks: 77 done / 122 total; 36 can run I1018 05:14:50.390037 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.390371 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.390559 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.391453 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.391850 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.393262 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.393380 33299 executor.go:111] Tasks: 113 done / 122 total; 5 can run I1018 05:14:50.393863 33299 executor.go:111] Tasks: 118 done / 122 total; 4 can run I1018 05:14:50.394118 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:14:50.469093 33299 target.go:112] Terraform output is in /tmp/test344830745/out --- PASS: TestBastionAdditionalUserData (0.27s) === RUN TestPrivateWeave I1018 05:14:50.474528 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:50.474828 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:50.474919 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:50.475018 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:50.475094 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:50.475201 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:50.475233 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:50.475272 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:50.475306 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:50.507455 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:50.510858 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.510955 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.511029 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.551674 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:50.669337 33299 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1018 05:14:50.669629 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:50.669814 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:50.670145 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:50.670372 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:50.670460 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:50.670424 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:50.670584 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:50.673110 33299 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1018 05:14:50.681527 33299 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1018 05:14:50.681728 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.681871 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.681979 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.682139 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.683329 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.683571 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.683821 33299 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1018 05:14:50.684576 33299 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1018 05:14:50.684852 33299 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1018 05:14:50.762232 33299 target.go:112] Terraform output is in /tmp/test602309028/out --- PASS: TestPrivateWeave (0.29s) === RUN TestPrivateFlannel I1018 05:14:50.766859 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:50.767101 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:50.767169 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:50.767275 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:50.767338 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:50.767391 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:50.767426 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:50.767469 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:50.767503 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:50.797456 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:50.800969 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.801075 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.801141 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:50.840082 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:51.036469 33299 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1018 05:14:51.037017 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:51.037252 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.037251 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:51.037436 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.037447 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:51.037495 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:51.037628 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:51.044265 33299 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1018 05:14:51.049905 33299 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1018 05:14:51.050499 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.050708 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.050980 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.051150 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.051969 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.053478 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.053727 33299 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1018 05:14:51.054105 33299 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1018 05:14:51.054408 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:14:51.131786 33299 target.go:112] Terraform output is in /tmp/test494733231/out --- PASS: TestPrivateFlannel (0.37s) === RUN TestPrivateCalico I1018 05:14:51.136687 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:51.136916 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:51.136978 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:51.137083 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:51.137138 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:51.137182 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:51.137215 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:51.137253 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:51.137282 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:51.166399 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:51.169996 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.170087 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.170146 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.209564 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:51.671780 33299 executor.go:111] Tasks: 0 done / 127 total; 50 can run W1018 05:14:51.672280 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:51.672872 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.672892 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:51.672921 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.672921 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:51.672991 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:51.673023 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:51.675250 33299 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1018 05:14:51.680827 33299 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1018 05:14:51.681094 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.681274 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.681637 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.681766 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.683452 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.687953 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.688095 33299 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1018 05:14:51.688658 33299 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1018 05:14:51.688987 33299 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1018 05:14:51.764441 33299 target.go:112] Terraform output is in /tmp/test1542605765/out --- PASS: TestPrivateCalico (0.63s) === RUN TestPrivateCilium I1018 05:14:51.770129 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:51.770358 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:51.770419 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:51.770528 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:51.770583 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:51.770630 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:51.770660 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:51.770701 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:51.770728 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:51.799372 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:51.802997 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.803088 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.803150 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.842277 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:51.980221 33299 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1018 05:14:51.980924 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.980926 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:51.980975 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:51.980988 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:51.981498 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:51.981713 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:51.981731 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:51.984095 33299 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1018 05:14:51.989406 33299 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1018 05:14:51.989694 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.990081 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.990285 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.990449 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.992487 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.996220 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:51.996370 33299 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1018 05:14:51.996827 33299 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1018 05:14:51.997108 33299 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1018 05:14:52.072491 33299 target.go:112] Terraform output is in /tmp/test1060293051/out I1018 05:14:52.076829 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:52.077044 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:52.077104 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:52.077157 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:52.077211 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:52.077255 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:52.077283 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:52.077321 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:52.077349 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:52.105898 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:52.109801 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.109886 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.109939 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.149745 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:52.298081 33299 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1018 05:14:52.298585 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:52.298803 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:52.298844 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:52.298948 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:52.299018 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:52.299185 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:52.299317 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:52.299449 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:52.299467 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:52.301011 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1018 05:14:52.304982 33299 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1018 05:14:52.310468 33299 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1018 05:14:52.310927 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.311149 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.311454 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.311667 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.312354 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.313642 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.313784 33299 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1018 05:14:52.314186 33299 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1018 05:14:52.314433 33299 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1018 05:14:52.319172 33299 target.go:168] Cloudformation output is in /tmp/test1637436785/out --- PASS: TestPrivateCilium (0.56s) === RUN TestPrivateCilium2 I1018 05:14:52.332278 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:52.332493 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:52.332546 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:52.332630 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:52.332685 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:52.332727 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:52.332756 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:52.332794 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:52.332824 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:52.366229 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:52.370049 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.370150 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.370215 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:52.409606 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:53.060993 33299 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1018 05:14:53.061302 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:53.061320 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:53.061419 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:53.061449 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:53.061622 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:53.061903 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:53.062054 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:53.065600 33299 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1018 05:14:53.074775 33299 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1018 05:14:53.075089 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.075306 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.075498 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.075564 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.076825 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.077845 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.077968 33299 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1018 05:14:53.078376 33299 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1018 05:14:53.078674 33299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1018 05:14:53.156304 33299 target.go:112] Terraform output is in /tmp/test3951446435/out I1018 05:14:53.163844 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:53.164072 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:53.164138 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:53.164191 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:53.164248 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:53.164300 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:53.164333 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:53.164373 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:53.164404 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:53.192655 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:53.196154 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.196238 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.196296 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.237039 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:53.888470 33299 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1018 05:14:53.888987 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:53.889259 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:53.889380 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:53.889516 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:53.889660 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:53.889751 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:53.889694 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:53.889860 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:53.889800 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:53.889950 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I1018 05:14:53.895882 33299 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1018 05:14:53.902046 33299 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1018 05:14:53.903820 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.904063 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.904310 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.904686 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.908628 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.910026 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.910237 33299 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1018 05:14:53.910729 33299 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1018 05:14:53.911078 33299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1018 05:14:53.915474 33299 target.go:168] Cloudformation output is in /tmp/test3329396280/out --- PASS: TestPrivateCilium2 (1.60s) === RUN TestPrivateCiliumAdvanced I1018 05:14:53.930978 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:53.931202 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:53.931255 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:53.931349 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:53.931401 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:53.931456 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:53.931488 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:53.931525 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:53.931554 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:53.964553 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:53.967729 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.967817 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:53.967872 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.008093 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:54.135130 33299 executor.go:111] Tasks: 0 done / 126 total; 53 can run I1018 05:14:54.135473 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:54.135509 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:54.135656 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:54.135759 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:54.135761 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:54.135798 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:54.136072 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:54.138293 33299 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1018 05:14:54.142191 33299 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1018 05:14:54.142680 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.142805 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.142898 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.142958 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.143460 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.145432 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.145556 33299 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1018 05:14:54.146038 33299 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1018 05:14:54.146308 33299 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1018 05:14:54.214440 33299 target.go:112] Terraform output is in /tmp/test4252576333/out I1018 05:14:54.217237 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:54.217387 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:54.217432 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:54.217465 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:54.217505 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:54.217537 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:54.217554 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:54.217578 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:54.217597 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:54.237930 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:54.240147 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.240213 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.240251 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.279145 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:54.405352 33299 executor.go:111] Tasks: 0 done / 126 total; 53 can run W1018 05:14:54.405634 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:54.405779 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:14:54.405825 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:54.405939 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:14:54.406127 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:54.406167 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:54.406244 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:54.406290 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:54.406309 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:54.406408 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I1018 05:14:54.408046 33299 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1018 05:14:54.415942 33299 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1018 05:14:54.416381 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.416653 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.416833 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.416939 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.417600 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.418207 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.418313 33299 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1018 05:14:54.418793 33299 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1018 05:14:54.419075 33299 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1018 05:14:54.421591 33299 target.go:168] Cloudformation output is in /tmp/test1412359886/out --- PASS: TestPrivateCiliumAdvanced (0.50s) === RUN TestPrivateCanal I1018 05:14:54.429606 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:54.429771 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:54.429810 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:54.429873 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:54.429913 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:54.429942 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:54.429958 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:54.429985 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:54.430007 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:54.446848 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:14:54.448908 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.448972 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.449010 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.487526 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:54.871464 33299 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1018 05:14:54.871818 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:54.871975 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:54.872046 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:54.872118 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:54.872133 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:54.872199 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:54.872481 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:54.874853 33299 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1018 05:14:54.878209 33299 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1018 05:14:54.878477 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.879750 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.880884 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.881113 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.882662 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.884391 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:54.884492 33299 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1018 05:14:54.884953 33299 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1018 05:14:54.885207 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:14:54.945409 33299 target.go:112] Terraform output is in /tmp/test1951364834/out --- PASS: TestPrivateCanal (0.52s) === RUN TestPrivateKopeio I1018 05:14:54.950832 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:54.951066 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:54.951127 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:54.951177 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:54.951231 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:54.951329 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:54.951359 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:54.951396 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:54.951426 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:54.980903 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:54.984779 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:54.984878 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:54.984965 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.025332 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:55.137954 33299 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1018 05:14:55.138223 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:55.138249 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:55.138365 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:55.138549 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:55.138615 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:55.138764 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.138801 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:55.145409 33299 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1018 05:14:55.151404 33299 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1018 05:14:55.151641 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.151832 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.151908 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.152056 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.153341 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.154978 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.155108 33299 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1018 05:14:55.155755 33299 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1018 05:14:55.156014 33299 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1018 05:14:55.236466 33299 target.go:112] Terraform output is in /tmp/test1811663942/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I1018 05:14:55.240991 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:55.241233 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:55.241298 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:55.241351 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:55.241408 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:55.241502 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:55.241533 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:55.241571 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:55.241620 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:55.271631 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:55.275025 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.275110 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.275177 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.316499 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:55.420536 33299 executor.go:111] Tasks: 0 done / 108 total; 45 can run I1018 05:14:55.420901 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:55.422592 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:55.422636 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.422674 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:55.422804 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:55.423185 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:55.423578 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.428092 33299 executor.go:111] Tasks: 45 done / 108 total; 25 can run I1018 05:14:55.433647 33299 executor.go:111] Tasks: 70 done / 108 total; 31 can run I1018 05:14:55.434118 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.434319 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.434407 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.434645 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.435601 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.437025 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.437159 33299 executor.go:111] Tasks: 101 done / 108 total; 4 can run I1018 05:14:55.437651 33299 executor.go:111] Tasks: 105 done / 108 total; 3 can run I1018 05:14:55.437960 33299 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1018 05:14:55.512837 33299 target.go:112] Terraform output is in /tmp/test741986990/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I1018 05:14:55.517056 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:55.517290 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:55.517350 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:55.517403 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:55.517460 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:55.517554 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:55.517583 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:55.517622 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:55.517654 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:55.546444 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:55.553097 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.553208 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.553282 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.595476 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:55.703903 33299 executor.go:111] Tasks: 0 done / 107 total; 45 can run W1018 05:14:55.704312 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.704339 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:55.704548 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.704718 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:14:55.704792 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.704753 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:55.704861 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:55.707475 33299 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1018 05:14:55.707921 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:55.712829 33299 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1018 05:14:55.713291 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.713494 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.713706 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.713981 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.714811 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.715755 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.715888 33299 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1018 05:14:55.716315 33299 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1018 05:14:55.716656 33299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1018 05:14:55.785919 33299 target.go:112] Terraform output is in /tmp/test1004914838/out --- PASS: TestPrivateSharedSubnet (0.28s) === RUN TestPrivateSharedIP I1018 05:14:55.793108 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:55.793391 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:55.793459 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:55.793620 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:55.793725 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:55.793803 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:55.793848 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:55.793920 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:55.793976 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:55.821998 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:55.825427 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.825528 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.825591 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.865584 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:55.967014 33299 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1018 05:14:55.967600 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:55.967691 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.967824 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:55.967859 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:55.967978 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:55.968055 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:55.968059 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:55.970425 33299 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1018 05:14:55.971104 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:55.975858 33299 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1018 05:14:55.976333 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.976512 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.976693 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.976770 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.977502 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.979086 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:55.979219 33299 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1018 05:14:55.979662 33299 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1018 05:14:55.979974 33299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1018 05:14:56.044585 33299 target.go:112] Terraform output is in /tmp/test38476191/out I1018 05:14:56.048843 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:56.049061 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:56.049125 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:56.049176 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:56.049234 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:56.049283 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:56.049311 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:56.049349 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:56.049379 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:56.077329 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:56.080575 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.080669 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.080728 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.120741 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:14:56.223174 33299 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1018 05:14:56.223610 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:56.223814 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:14:56.223843 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:56.224005 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:14:56.224235 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:14:56.224284 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:56.224308 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:14:56.224347 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:14:56.224271 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:56.224403 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:14:56.225970 33299 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1018 05:14:56.226577 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:56.231451 33299 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1018 05:14:56.231714 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.231844 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.232206 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.232334 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.232954 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.236003 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:14:56.237005 33299 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1018 05:14:56.238367 33299 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1018 05:14:56.238651 33299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1018 05:14:56.242458 33299 target.go:168] Cloudformation output is in /tmp/test2501211098/out --- PASS: TestPrivateSharedIP (0.46s) === RUN TestPrivateDns1 I1018 05:14:56.255610 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:56.255921 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:56.256004 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:56.256088 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:56.256186 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:56.256299 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:56.256355 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:56.256419 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:56.256471 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:56.287760 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.287854 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.287922 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.448205 33299 executor.go:111] Tasks: 0 done / 120 total; 46 can run W1018 05:14:56.449047 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:56.449224 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:56.449294 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:56.449342 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:56.455969 33299 executor.go:111] Tasks: 46 done / 120 total; 24 can run I1018 05:14:56.456453 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I1018 05:14:56.456494 33299 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I1018 05:14:56.456578 33299 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I1018 05:14:56.456598 33299 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I1018 05:14:56.456727 33299 executor.go:111] Tasks: 70 done / 120 total; 35 can run I1018 05:14:56.457287 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.457538 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.460283 33299 executor.go:111] Tasks: 105 done / 120 total; 5 can run W1018 05:14:56.460366 33299 api.go:53] PageSize not implemented W1018 05:14:56.460450 33299 api.go:53] PageSize not implemented I1018 05:14:56.466142 33299 executor.go:111] Tasks: 110 done / 120 total; 6 can run I1018 05:14:56.466406 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.466643 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.467777 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.469042 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.469172 33299 executor.go:111] Tasks: 116 done / 120 total; 2 can run I1018 05:14:56.469519 33299 executor.go:111] Tasks: 118 done / 120 total; 2 can run I1018 05:14:56.469716 33299 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1018 05:14:56.552446 33299 target.go:112] Terraform output is in /tmp/test407260487/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I1018 05:14:56.556960 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:56.557199 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:56.557263 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:56.557315 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:56.557371 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:56.557468 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:56.557497 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:56.557539 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:56.557570 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:56.593335 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.593439 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.593506 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.734540 33299 executor.go:111] Tasks: 0 done / 116 total; 44 can run W1018 05:14:56.734807 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:56.735297 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:56.735444 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:56.735569 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:56.737522 33299 executor.go:111] Tasks: 44 done / 116 total; 22 can run I1018 05:14:56.737951 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I1018 05:14:56.737994 33299 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I1018 05:14:56.738075 33299 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I1018 05:14:56.738097 33299 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I1018 05:14:56.738087 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:14:56.738236 33299 executor.go:111] Tasks: 66 done / 116 total; 35 can run I1018 05:14:56.738739 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.738916 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.741953 33299 executor.go:111] Tasks: 101 done / 116 total; 5 can run W1018 05:14:56.742035 33299 api.go:53] PageSize not implemented W1018 05:14:56.742281 33299 api.go:53] PageSize not implemented I1018 05:14:56.747285 33299 executor.go:111] Tasks: 106 done / 116 total; 6 can run I1018 05:14:56.747547 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.747701 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.748943 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.749834 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.749962 33299 executor.go:111] Tasks: 112 done / 116 total; 2 can run I1018 05:14:56.750327 33299 executor.go:111] Tasks: 114 done / 116 total; 2 can run I1018 05:14:56.750534 33299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1018 05:14:56.829124 33299 target.go:112] Terraform output is in /tmp/test3079115461/out --- PASS: TestPrivateDns2 (0.28s) === RUN TestDiscoveryFeatureGate I1018 05:14:56.833328 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:56.833555 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:56.833615 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:56.833721 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:56.833778 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:56.833824 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:56.833854 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:56.833892 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:56.833921 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:56.860002 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:14:56.863422 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.863519 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:56.903609 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:57.104359 33299 executor.go:111] Tasks: 0 done / 99 total; 51 can run W1018 05:14:57.104886 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:57.104960 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:57.104992 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:57.105074 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:14:57.106486 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:57.108455 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:57.112501 33299 executor.go:111] Tasks: 51 done / 99 total; 23 can run I1018 05:14:57.120091 33299 executor.go:111] Tasks: 74 done / 99 total; 21 can run I1018 05:14:57.120372 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.120617 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.121732 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.122732 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.122824 33299 executor.go:111] Tasks: 95 done / 99 total; 2 can run I1018 05:14:57.123068 33299 executor.go:111] Tasks: 97 done / 99 total; 2 can run I1018 05:14:57.123274 33299 executor.go:111] Tasks: 99 done / 99 total; 0 can run I1018 05:14:57.184558 33299 target.go:112] Terraform output is in /tmp/test1660779314/out --- PASS: TestDiscoveryFeatureGate (0.36s) === RUN TestVFSServiceAccountIssuerDiscovery I1018 05:14:57.189922 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:57.190170 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:57.190230 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:57.190284 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:57.190339 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:57.190438 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:57.190468 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:57.190506 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:57.190534 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:57.215364 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:57.218551 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.218645 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.258664 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:57.359656 33299 executor.go:111] Tasks: 0 done / 90 total; 45 can run I1018 05:14:57.359975 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:57.360664 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:57.360711 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:57.360829 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:57.360853 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:57.360854 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:57.366261 33299 executor.go:111] Tasks: 45 done / 90 total; 20 can run I1018 05:14:57.371433 33299 executor.go:111] Tasks: 65 done / 90 total; 21 can run I1018 05:14:57.371902 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.372057 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.373152 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.374384 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.374573 33299 executor.go:111] Tasks: 86 done / 90 total; 2 can run I1018 05:14:57.375017 33299 executor.go:111] Tasks: 88 done / 90 total; 2 can run I1018 05:14:57.375182 33299 executor.go:111] Tasks: 90 done / 90 total; 0 can run I1018 05:14:57.433890 33299 target.go:112] Terraform output is in /tmp/test2183790843/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.25s) === RUN TestAWSLBController I1018 05:14:57.438509 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:57.438738 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:57.438799 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:57.438851 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:57.438907 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:57.438996 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:57.439027 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:57.439065 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:57.439094 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:57.464461 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:57.467770 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.467879 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:57.508515 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:58.098528 33299 executor.go:111] Tasks: 0 done / 96 total; 49 can run I1018 05:14:58.098940 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:58.098992 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:58.099098 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:58.099478 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:58.100497 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:14:58.099475 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:58.107124 33299 executor.go:111] Tasks: 49 done / 96 total; 22 can run I1018 05:14:58.112540 33299 executor.go:111] Tasks: 71 done / 96 total; 21 can run I1018 05:14:58.113063 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.114338 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.118198 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.119800 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.119945 33299 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1018 05:14:58.120370 33299 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1018 05:14:58.120589 33299 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1018 05:14:58.175132 33299 target.go:112] Terraform output is in /tmp/test1600176092/out --- PASS: TestAWSLBController (0.75s) === RUN TestManyAddons I1018 05:14:58.183770 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:58.184013 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:58.184077 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:58.184133 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:58.184192 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:58.184289 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:58.184320 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:58.184360 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:58.184391 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:58.218357 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:58.222184 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.222287 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:58.261381 33299 zones.go:93] ListHostedZonesPages { } I1018 05:14:59.032411 33299 executor.go:111] Tasks: 0 done / 95 total; 50 can run I1018 05:14:59.033845 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:14:59.033920 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:14:59.034051 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:14:59.034395 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:14:59.035524 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:14:59.037411 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:14:59.042353 33299 executor.go:111] Tasks: 50 done / 95 total; 19 can run I1018 05:14:59.052249 33299 executor.go:111] Tasks: 69 done / 95 total; 22 can run I1018 05:14:59.052932 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.053186 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.054146 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.055964 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.056088 33299 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1018 05:14:59.056425 33299 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1018 05:14:59.056647 33299 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1018 05:14:59.112185 33299 target.go:112] Terraform output is in /tmp/test3463161082/out --- PASS: TestManyAddons (0.94s) === RUN TestManyAddonsCCMIRSA I1018 05:14:59.120868 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:59.121115 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:59.121178 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:59.121286 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:59.121341 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:59.121386 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:59.121419 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:59.121459 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:59.121489 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:59.149373 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:14:59.153193 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.153287 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:14:59.192345 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:00.047643 33299 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1018 05:15:00.048559 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:00.048606 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:00.048681 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:00.048741 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:00.048699 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:00.048824 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:00.051485 33299 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1018 05:15:00.061840 33299 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1018 05:15:00.062309 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.062537 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.063690 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.064839 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.064994 33299 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1018 05:15:00.065319 33299 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1018 05:15:00.065555 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:00.136426 33299 target.go:112] Terraform output is in /tmp/test638448228/out --- PASS: TestManyAddonsCCMIRSA (1.02s) === RUN TestManyAddonsCCMIRSA23 I1018 05:15:00.145534 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:00.145770 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:00.145827 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:00.145930 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:00.145982 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:00.146027 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:00.146060 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:00.146099 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:00.146127 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:00.173164 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1018 05:15:00.176636 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.176724 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:00.215995 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:01.079201 33299 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1018 05:15:01.079969 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:01.079981 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:01.081821 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:01.082878 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:01.082946 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:01.083093 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:01.088716 33299 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1018 05:15:01.095485 33299 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1018 05:15:01.095717 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.096081 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.097190 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.098347 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.098473 33299 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1018 05:15:01.098775 33299 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1018 05:15:01.098971 33299 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1018 05:15:01.170769 33299 target.go:112] Terraform output is in /tmp/test3078242061/out --- PASS: TestManyAddonsCCMIRSA23 (1.03s) === RUN TestManyAddonsCCMIRSA24 I1018 05:15:01.180259 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:01.180498 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:01.180560 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:01.180664 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:01.180719 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:01.180769 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:01.180798 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:01.180837 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:01.180867 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:01.208151 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:01.211744 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.211833 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:01.252627 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:02.136947 33299 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1018 05:15:02.137423 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:02.138768 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:02.138823 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:02.138958 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:02.142156 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:02.143043 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:02.145681 33299 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1018 05:15:02.155672 33299 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1018 05:15:02.156055 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.156318 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.157392 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.158887 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.159019 33299 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1018 05:15:02.159348 33299 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1018 05:15:02.159555 33299 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1018 05:15:02.226050 33299 target.go:112] Terraform output is in /tmp/test3769011190/out --- PASS: TestManyAddonsCCMIRSA24 (1.06s) === RUN TestManyAddonsCCMIRSA25 I1018 05:15:02.236591 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:02.236828 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:02.236890 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:02.236996 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:02.237051 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:02.237097 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:02.237128 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:02.237167 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:02.237197 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:02.262472 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:15:02.265409 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.265484 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:02.307554 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:03.176437 33299 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1018 05:15:03.176821 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:03.177338 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:03.178733 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:03.178787 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:03.178936 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:03.179409 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:03.185928 33299 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1018 05:15:03.191559 33299 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1018 05:15:03.192242 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.192560 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.193526 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.195684 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.195825 33299 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1018 05:15:03.196146 33299 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1018 05:15:03.196417 33299 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1018 05:15:03.267357 33299 target.go:112] Terraform output is in /tmp/test2914780729/out --- PASS: TestManyAddonsCCMIRSA25 (1.04s) === RUN TestManyAddonsCCMIRSA26 I1018 05:15:03.276914 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:03.277153 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:03.277214 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:03.277265 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:03.277322 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:03.277415 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:03.277445 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:03.277483 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:03.277512 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:03.304853 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:15:03.308293 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.308399 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:03.350438 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:04.184096 33299 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1018 05:15:04.184322 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:04.184469 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:04.185833 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:04.185883 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:04.185989 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:04.187256 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:04.193651 33299 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1018 05:15:04.204529 33299 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1018 05:15:04.204804 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.205098 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.206320 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.207642 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.207773 33299 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1018 05:15:04.208111 33299 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1018 05:15:04.208345 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:04.276997 33299 target.go:112] Terraform output is in /tmp/test3632234329/out --- PASS: TestManyAddonsCCMIRSA26 (1.01s) === RUN TestCCM I1018 05:15:04.286556 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:04.286806 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:04.286868 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:04.286923 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:04.286978 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:04.287073 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:04.287102 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:04.287141 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:04.287172 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:04.313827 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:04.317240 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.317326 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:04.357659 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:05.156912 33299 executor.go:111] Tasks: 0 done / 96 total; 51 can run W1018 05:15:05.157465 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:05.157777 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:05.157807 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:05.157925 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:05.158008 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:05.158112 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:05.165009 33299 executor.go:111] Tasks: 51 done / 96 total; 19 can run I1018 05:15:05.170910 33299 executor.go:111] Tasks: 70 done / 96 total; 22 can run I1018 05:15:05.171406 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.171546 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.172609 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.174059 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.174176 33299 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1018 05:15:05.174464 33299 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1018 05:15:05.174640 33299 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1018 05:15:05.228793 33299 target.go:112] Terraform output is in /tmp/test2886018529/out --- PASS: TestCCM (0.95s) === RUN TestExternalDNS I1018 05:15:05.241253 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:05.241509 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:05.241567 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:05.241685 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:05.241741 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:05.241785 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:05.241816 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:05.241853 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:05.241882 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:05.268065 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:05.271211 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.271317 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.311139 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:05.415420 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:15:05.416441 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:05.416504 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:05.416556 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:05.416564 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:05.416730 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:05.416845 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:05.418218 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:05.427718 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:05.428028 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.428217 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.429434 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.430529 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.430647 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:05.430998 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:05.431204 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:05.482400 33299 target.go:112] Terraform output is in /tmp/test4229909139/out I1018 05:15:05.489820 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:05.490085 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:05.490152 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:05.490208 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:05.490268 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:05.490324 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:05.490355 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:05.490399 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:05.490430 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:05.515349 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:05.518448 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.518549 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.558510 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:05.662554 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:15:05.663020 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:05.663050 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:05.663168 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:15:05.663315 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:05.663575 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:05.663619 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:05.663770 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:05.663788 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:05.663883 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1018 05:15:05.666261 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:05.671741 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:05.672168 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.672411 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.673506 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.674568 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.674708 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:05.675046 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:05.675237 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:05.678133 33299 target.go:168] Cloudformation output is in /tmp/test242628616/out --- PASS: TestExternalDNS (0.45s) === RUN TestExternalDNSIRSA I1018 05:15:05.689001 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:05.689359 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:05.689471 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:05.689574 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:05.689710 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:05.689922 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:05.690130 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:05.690844 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:05.690882 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:05.717568 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:05.720720 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.720818 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.761382 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:05.879410 33299 executor.go:111] Tasks: 0 done / 92 total; 46 can run I1018 05:15:05.879743 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:05.879884 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:05.880467 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:05.880603 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:05.880644 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:05.880748 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:05.882734 33299 executor.go:111] Tasks: 46 done / 92 total; 21 can run I1018 05:15:05.887973 33299 executor.go:111] Tasks: 67 done / 92 total; 21 can run I1018 05:15:05.888332 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.888671 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.891291 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.895838 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.895997 33299 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1018 05:15:05.896335 33299 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1018 05:15:05.896612 33299 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1018 05:15:05.948961 33299 target.go:112] Terraform output is in /tmp/test2957956834/out --- PASS: TestExternalDNSIRSA (0.26s) === RUN TestKarpenter I1018 05:15:05.953379 33299 featureflag.go:160] FeatureFlag "Karpenter"=true I1018 05:15:05.953429 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:05.953648 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:05.953712 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:05.953767 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:05.953818 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:05.953908 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:05.953939 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:05.953976 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:05.954005 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:05.989125 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:15:05.992796 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.992895 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:05.992958 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.031980 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:06.219556 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.219704 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.219833 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.339295 33299 executor.go:111] Tasks: 0 done / 108 total; 53 can run I1018 05:15:06.339859 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:06.340145 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:06.340161 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:06.340510 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:06.340541 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:06.340651 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:06.343129 33299 executor.go:111] Tasks: 53 done / 108 total; 26 can run I1018 05:15:06.350093 33299 executor.go:111] Tasks: 79 done / 108 total; 25 can run I1018 05:15:06.350576 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.350765 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.350997 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.351268 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.353289 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.353541 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.354590 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.355971 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.356128 33299 executor.go:111] Tasks: 104 done / 108 total; 2 can run I1018 05:15:06.357926 33299 executor.go:111] Tasks: 106 done / 108 total; 2 can run I1018 05:15:06.358196 33299 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1018 05:15:06.429930 33299 target.go:112] Terraform output is in /tmp/test3570100072/out I1018 05:15:06.436240 33299 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.48s) === RUN TestSharedSubnet I1018 05:15:06.436380 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:06.436629 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:06.436692 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:06.436747 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:06.436802 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:06.436893 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:06.436922 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:06.436959 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:06.436996 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:06.461502 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:06.464607 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.464705 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.506039 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:06.604300 33299 executor.go:111] Tasks: 0 done / 80 total; 42 can run W1018 05:15:06.604712 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:06.604697 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:06.604995 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:06.605316 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:06.605358 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:06.605479 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:06.607481 33299 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1018 05:15:06.607928 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:06.612680 33299 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1018 05:15:06.613172 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.613372 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.614233 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.615703 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.615820 33299 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1018 05:15:06.616123 33299 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1018 05:15:06.616376 33299 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1018 05:15:06.665491 33299 target.go:112] Terraform output is in /tmp/test3860467034/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I1018 05:15:06.669445 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:06.669692 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:06.669760 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:06.669814 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:06.669869 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:06.669965 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:06.669995 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:06.670032 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:06.670061 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:06.694394 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:06.697544 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.697639 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.738874 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:06.838105 33299 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1018 05:15:06.838738 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:06.838739 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:06.838784 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:06.838887 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:06.838970 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:06.839088 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:06.841578 33299 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1018 05:15:06.842115 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:06.846708 33299 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1018 05:15:06.847078 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.847341 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.848180 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.849774 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:06.849905 33299 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1018 05:15:06.850237 33299 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1018 05:15:06.850474 33299 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1018 05:15:06.903604 33299 target.go:112] Terraform output is in /tmp/test510684592/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I1018 05:15:06.907529 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:06.907769 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:06.907831 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:06.907884 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:06.907943 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:06.908037 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:06.908067 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:06.908104 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:06.908137 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:06.940688 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:06.944071 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:06.944162 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:06.944220 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:06.944276 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:06.986471 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:07.093902 33299 executor.go:111] Tasks: 0 done / 101 total; 51 can run W1018 05:15:07.094263 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:07.094421 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:07.096654 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:07.096713 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:07.096854 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:07.096994 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:07.097217 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:07.098509 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:07.101764 33299 executor.go:111] Tasks: 51 done / 101 total; 15 can run I1018 05:15:07.107975 33299 executor.go:111] Tasks: 66 done / 101 total; 27 can run I1018 05:15:07.108311 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.108469 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.108591 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.108848 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.109815 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.110765 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.110924 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.111262 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:07.111383 33299 executor.go:111] Tasks: 93 done / 101 total; 4 can run I1018 05:15:07.112920 33299 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1018 05:15:07.113300 33299 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1018 05:15:07.189780 33299 target.go:112] Terraform output is in /tmp/test1148358300/out --- PASS: TestExistingIAM (0.29s) === RUN TestPhaseNetwork I1018 05:15:07.194975 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:07.195228 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:07.195292 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:07.195345 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:07.195402 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:07.195448 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:07.195479 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:07.195518 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:07.195547 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:07.227808 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:07.231129 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.231623 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.231699 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.271458 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:07.378526 33299 executor.go:111] Tasks: 0 done / 119 total; 46 can run I1018 05:15:07.378827 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:07.378951 33299 executor.go:111] Tasks: 46 done / 119 total; 28 can run I1018 05:15:07.379534 33299 executor.go:111] Tasks: 74 done / 119 total; 36 can run I1018 05:15:07.379940 33299 executor.go:111] Tasks: 110 done / 119 total; 5 can run I1018 05:15:07.380168 33299 executor.go:111] Tasks: 115 done / 119 total; 4 can run I1018 05:15:07.380317 33299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I1018 05:15:07.389072 33299 target.go:112] Terraform output is in /tmp/test2026779944/out --- PASS: TestPhaseNetwork (0.19s) === RUN TestExternalLoadBalancer I1018 05:15:07.389866 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:07.390089 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:07.390145 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:07.390244 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:07.390300 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:07.390346 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:07.390374 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:07.390413 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:07.390440 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:07.418895 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:07.422302 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.422480 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.462461 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:07.563976 33299 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1018 05:15:07.564494 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:07.564746 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:07.564781 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:07.564893 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:07.564897 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:07.565204 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:07.567558 33299 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1018 05:15:07.574836 33299 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1018 05:15:07.575081 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.575385 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.576323 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.577909 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.578028 33299 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1018 05:15:07.578382 33299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1018 05:15:07.578628 33299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1018 05:15:07.632519 33299 target.go:112] Terraform output is in /tmp/test3381127685/out I1018 05:15:07.636487 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:07.636702 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:07.636767 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:07.636822 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:07.636878 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:07.636923 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:07.636952 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:07.636990 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:07.637020 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:07.662258 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:07.665462 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.665559 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.707783 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:07.810406 33299 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1018 05:15:07.810829 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:07.810867 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:07.810969 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:07.811140 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:07.811270 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:07.811288 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:07.811381 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W1018 05:15:07.811544 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:07.811649 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:07.813370 33299 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1018 05:15:07.818522 33299 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1018 05:15:07.819062 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.819290 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.820433 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.821321 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.821462 33299 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1018 05:15:07.821751 33299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1018 05:15:07.821956 33299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1018 05:15:07.826253 33299 target.go:168] Cloudformation output is in /tmp/test2325429692/out --- PASS: TestExternalLoadBalancer (0.45s) === RUN TestPhaseIAM integration_test.go:853: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:861: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I1018 05:15:07.838195 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:07.838450 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:07.838501 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:07.838596 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:07.838647 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:07.838690 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:07.838719 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:07.838757 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:07.838787 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:07.870402 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:07.873230 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873308 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873356 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873401 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873440 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873479 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.873521 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:07.914768 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:08.017289 33299 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1018 05:15:08.017473 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.017597 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:08.017772 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:15:08.017855 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.017824 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:08.017994 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:08.017795 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:08.018147 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.019946 33299 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1018 05:15:08.029105 33299 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1018 05:15:08.029322 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.029524 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.029632 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.029847 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.031185 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.031395 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.032175 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.032363 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.032493 33299 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1018 05:15:08.032858 33299 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1018 05:15:08.033145 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:08.114956 33299 target.go:112] Terraform output is in /tmp/test605496603/out I1018 05:15:08.120115 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:08.120360 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:08.120425 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:08.120478 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:08.120535 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:08.120581 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:08.120608 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:08.120647 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:08.120676 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:08.149151 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:08.151736 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.151806 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.151848 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.151889 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.151928 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.151971 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.152006 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.192657 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:08.302277 33299 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1018 05:15:08.302504 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:08.302522 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:08.302605 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:15:08.304246 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:08.304659 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:08.304849 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:08.304800 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:08.305040 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:08.305067 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:08.305252 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } W1018 05:15:08.307597 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:08.310653 33299 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1018 05:15:08.316645 33299 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1018 05:15:08.316889 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.317049 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.317219 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.317383 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.318637 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.320927 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.321099 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.321621 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.321994 33299 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1018 05:15:08.323096 33299 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1018 05:15:08.324127 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:08.329016 33299 target.go:168] Cloudformation output is in /tmp/test3712290646/out --- PASS: TestMixedInstancesASG (0.51s) === RUN TestMixedInstancesSpotASG I1018 05:15:08.345893 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:08.346107 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:08.346160 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:08.346257 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:08.346311 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:08.346355 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:08.346383 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:08.346425 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:08.346452 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:08.378369 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:08.381253 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381331 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381381 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381430 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381474 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381515 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.381555 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.420482 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:08.534157 33299 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1018 05:15:08.534814 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:15:08.534877 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.534848 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:08.534990 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:08.535045 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:08.535053 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.535130 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:08.535259 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:08.537913 33299 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1018 05:15:08.549984 33299 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1018 05:15:08.550360 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.550582 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.550797 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.551075 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.552099 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.553250 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.555662 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.555923 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.556064 33299 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1018 05:15:08.556510 33299 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1018 05:15:08.556846 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:08.638749 33299 target.go:112] Terraform output is in /tmp/test2841335959/out I1018 05:15:08.644047 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:08.644258 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:08.644324 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:08.644377 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:08.644432 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:08.644478 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:08.644507 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:08.644543 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:08.644573 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:08.677627 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:08.680752 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.680843 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.680903 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.680956 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.681002 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.681046 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.681087 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.723273 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:08.837179 33299 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1018 05:15:08.837614 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:08.837909 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:08.838094 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:08.838294 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:08.838324 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:08.838450 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:08.838473 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:08.838519 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:08.838481 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I1018 05:15:08.838563 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:08.838684 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:08.840589 33299 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1018 05:15:08.847446 33299 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1018 05:15:08.847700 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.847997 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.848166 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.848496 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.849465 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.850302 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.852702 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.852867 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:08.853197 33299 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1018 05:15:08.855082 33299 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1018 05:15:08.855497 33299 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1018 05:15:08.861515 33299 target.go:168] Cloudformation output is in /tmp/test1086768557/out --- PASS: TestMixedInstancesSpotASG (0.53s) === RUN TestAdditionalObjects I1018 05:15:08.878354 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:08.878584 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:08.878636 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:08.878685 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:08.878740 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:08.878822 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:08.878851 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:08.878888 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:08.878918 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:08.904756 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1018 05:15:08.907713 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:08.907798 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:08.948651 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:09.108541 33299 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1018 05:15:09.109239 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:09.109277 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:09.109414 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:09.109589 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:09.109640 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:09.109787 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:09.112137 33299 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1018 05:15:09.119761 33299 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1018 05:15:09.120169 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:09.120444 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:09.123674 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:09.127055 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1018 05:15:09.127197 33299 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1018 05:15:09.127537 33299 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1018 05:15:09.127744 33299 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1018 05:15:09.181648 33299 target.go:112] Terraform output is in /tmp/test2350539481/out --- PASS: TestAdditionalObjects (0.31s) === RUN TestContainerd I1018 05:15:09.185947 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:09.186184 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:09.186246 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:09.186359 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:09.186417 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:09.186464 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:09.186492 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:09.186531 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:09.186561 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:09.214910 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:09.218037 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.218136 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.258434 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:09.361092 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:15:09.361793 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:09.361830 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:09.361920 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W1018 05:15:09.362023 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:09.362056 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:09.362054 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:09.362225 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:09.362249 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:09.362344 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1018 05:15:09.364252 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:09.369127 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:09.369650 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.369880 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.370995 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.372024 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.372169 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:09.372437 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:09.372630 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:09.375493 33299 target.go:168] Cloudformation output is in /tmp/test2074693711/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I1018 05:15:09.384997 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:09.385216 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:09.385268 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:09.385351 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:09.385406 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:09.385446 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:09.385474 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:09.385514 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:09.385545 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:09.415442 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:09.419031 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.419130 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.458853 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:09.563758 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:15:09.564410 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:09.564618 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:09.564680 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:09.564663 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W1018 05:15:09.564779 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:09.564799 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:09.564907 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:09.564804 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:09.564977 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I1018 05:15:09.566545 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:09.570510 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:09.570750 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.570960 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.571645 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.573480 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.573627 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:09.573929 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:09.574139 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:09.577123 33299 target.go:168] Cloudformation output is in /tmp/test3047309927/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I1018 05:15:09.586970 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:09.587227 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:09.587279 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:09.587366 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:09.587422 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:09.587467 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:09.587497 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:09.587534 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:09.587565 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:09.616953 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:09.620361 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.620464 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.660169 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:09.758908 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1018 05:15:09.760550 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:09.761456 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:09.761581 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:09.761732 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:09.761593 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:09.761782 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:09.761944 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:09.761880 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } W1018 05:15:09.763126 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:09.766638 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:09.771974 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:09.772425 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.772690 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.774356 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.775274 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:09.775480 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:09.775898 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:09.776093 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:09.779161 33299 target.go:168] Cloudformation output is in /tmp/test954869058/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I1018 05:15:09.789222 33299 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1018 05:15:09.789264 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:09.789448 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:09.789496 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:09.789575 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:09.789634 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:09.789674 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:09.789700 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:09.789737 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:09.789762 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:09.820528 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1018 05:15:09.823840 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:09.823936 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:09.823997 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.366394 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:10.529348 33299 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1018 05:15:10.529697 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:10.530215 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:10.530249 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:10.530363 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:10.530367 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:10.530496 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:10.530411 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:10.530544 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:10.530517 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:10.530613 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:10.534562 33299 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1018 05:15:10.543356 33299 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1018 05:15:10.543670 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.543941 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.544172 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.545179 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.545497 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.546337 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.546481 33299 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1018 05:15:10.546779 33299 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1018 05:15:10.546984 33299 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1018 05:15:10.550825 33299 target.go:168] Cloudformation output is in /tmp/test341904005/out I1018 05:15:10.563146 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:10.563363 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:10.563419 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:10.563466 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:10.563519 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:10.563561 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:10.563589 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:10.563626 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:10.563652 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:10.594247 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1018 05:15:10.597534 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.597626 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.597693 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.639067 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:10.794706 33299 executor.go:111] Tasks: 0 done / 100 total; 47 can run I1018 05:15:10.795448 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:10.795482 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:10.795571 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:10.795584 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:10.795668 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:10.795693 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target W1018 05:15:10.795699 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:10.798547 33299 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1018 05:15:10.804517 33299 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1018 05:15:10.804760 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.805149 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.805371 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.806318 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.806714 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.807297 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.807426 33299 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1018 05:15:10.808657 33299 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1018 05:15:10.809001 33299 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1018 05:15:10.870548 33299 target.go:112] Terraform output is in /tmp/test547289881/out I1018 05:15:10.876362 33299 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.09s) === RUN TestNTHQueueProcessor I1018 05:15:10.876804 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:10.877190 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:10.877267 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:10.877381 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:10.877671 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:10.877739 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:10.877771 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:10.877817 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:10.877855 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:10.902959 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:10.906202 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:10.906299 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.479285 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:11.605445 33299 executor.go:111] Tasks: 0 done / 101 total; 45 can run W1018 05:15:11.605921 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:11.606096 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1018 05:15:11.606212 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:11.606148 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:11.606291 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:11.607130 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:11.609376 33299 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1018 05:15:11.619263 33299 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1018 05:15:11.619696 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.619886 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.620916 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.622470 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.622594 33299 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1018 05:15:11.622955 33299 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1018 05:15:11.623222 33299 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1018 05:15:11.686342 33299 target.go:112] Terraform output is in /tmp/test76747405/out I1018 05:15:11.691095 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:11.691306 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:11.691363 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:11.691412 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:11.691465 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:11.691508 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:11.691536 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:11.691576 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:11.691628 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:11.716177 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:11.719358 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.719447 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.760395 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I1018 05:15:11.881737 33299 executor.go:111] Tasks: 0 done / 101 total; 45 can run W1018 05:15:11.881987 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I1018 05:15:11.882473 33299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I1018 05:15:11.882508 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:11.882629 33299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I1018 05:15:11.882449 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:11.882651 33299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W1018 05:15:11.882768 33299 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I1018 05:15:11.882800 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:11.882904 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:11.886919 33299 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1018 05:15:11.895778 33299 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1018 05:15:11.896184 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.896430 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.897223 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.898657 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.898801 33299 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1018 05:15:11.899055 33299 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1018 05:15:11.899468 33299 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1018 05:15:11.904003 33299 target.go:168] Cloudformation output is in /tmp/test2614169944/out --- PASS: TestNTHQueueProcessor (1.04s) === RUN TestCustomIRSA I1018 05:15:11.915185 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:11.915414 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:11.915466 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:11.915513 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:11.915567 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:11.915648 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:11.915677 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:11.915714 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:11.915744 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:11.947869 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:11.951290 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.951393 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:11.990591 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:12.526454 33299 executor.go:111] Tasks: 0 done / 100 total; 50 can run I1018 05:15:12.526804 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:12.527092 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:12.529118 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:12.529153 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:12.529256 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1018 05:15:12.530462 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:12.535677 33299 executor.go:111] Tasks: 50 done / 100 total; 25 can run I1018 05:15:12.541138 33299 executor.go:111] Tasks: 75 done / 100 total; 21 can run I1018 05:15:12.541487 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.541768 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.542836 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.543914 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.544033 33299 executor.go:111] Tasks: 96 done / 100 total; 2 can run I1018 05:15:12.544426 33299 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1018 05:15:12.544757 33299 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1018 05:15:12.606995 33299 target.go:112] Terraform output is in /tmp/test2196695462/out --- PASS: TestCustomIRSA (0.70s) === RUN TestClusterNameDigit I1018 05:15:12.618284 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:12.618543 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:12.618608 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:12.618661 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:12.618718 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:12.618816 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:12.618850 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:12.618888 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:12.618917 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:12.647161 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:12.650457 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.650551 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.689544 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:12.797239 33299 executor.go:111] Tasks: 0 done / 95 total; 47 can run W1018 05:15:12.798013 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:12.798052 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1018 05:15:12.798123 33299 warmpool.go:124] ASG warm pool is not supported by the terraform target I1018 05:15:12.798296 33299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1018 05:15:12.798341 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:12.798480 33299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1018 05:15:12.800184 33299 executor.go:111] Tasks: 47 done / 95 total; 23 can run I1018 05:15:12.805387 33299 executor.go:111] Tasks: 70 done / 95 total; 21 can run I1018 05:15:12.805737 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.805898 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.810245 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.812936 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.813051 33299 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1018 05:15:12.813367 33299 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1018 05:15:12.813540 33299 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1018 05:15:12.867064 33299 target.go:112] Terraform output is in /tmp/test3470735525/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I1018 05:15:12.872047 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:12.872277 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:12.872340 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:12.872395 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:12.872453 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:12.872498 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:12.872527 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:12.872567 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:12.872597 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:12.872668 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I1018 05:15:12.901065 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:12.904095 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:12.904201 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:15:12.943854 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:13.038907 33299 iamrole.go:89] ListRoles: { } I1018 05:15:13.040472 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:15:13.040728 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.040925 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:13.041056 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:13.041215 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.041357 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I1018 05:15:13.041441 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:13.041442 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:13.041526 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1018 05:15:13.041566 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.041835 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.041866 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.041952 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I1018 05:15:13.042019 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.042065 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1018 05:15:13.042189 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:13.042319 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:13.042359 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.042492 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:13.042596 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:13.042645 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:13.042676 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:13.042698 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:13.042732 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.043358 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:13.043598 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.043759 33299 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1018 05:15:13.043778 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.043897 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1018 05:15:13.043904 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1018 05:15:13.043982 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1018 05:15:13.044030 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:13.044042 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:13.044171 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1018 05:15:13.044128 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1018 05:15:13.044383 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.044609 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1018 05:15:13.044679 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1018 05:15:13.044832 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:13.044965 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:13.045025 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:13.045073 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.045175 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:13.045312 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:13.045323 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.045560 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:13.045666 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I1018 05:15:13.045784 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:13.045855 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:13.045910 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:13.045963 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:13.046033 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:13.046081 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I1018 05:15:13.046188 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:13.046416 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1018 05:15:13.048940 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:13.049226 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.049409 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.049625 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.049829 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.049977 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.050115 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.050278 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:13.050348 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.050504 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.050582 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.050730 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.050872 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:13.050912 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.051052 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I1018 05:15:13.051200 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.051295 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1018 05:15:13.051427 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.051523 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.051619 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.051832 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.051975 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:13.052031 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1018 05:15:13.052161 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:13.052206 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1018 05:15:13.052341 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.054610 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:13.054881 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:13.055022 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:13.055072 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:13.055106 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:13.055171 33299 dns.go:238] Pre-creating DNS records I1018 05:15:13.055202 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:13.060567 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:13.067266 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.067386 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1018 05:15:13.107341 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:13.204745 33299 iamrole.go:89] ListRoles: { } I1018 05:15:13.206287 33299 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1018 05:15:13.206523 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:13.206624 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:13.206789 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I1018 05:15:13.206902 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:13.206947 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.207116 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.209130 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:13.209212 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:13.209334 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.209636 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.213364 33299 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1018 05:15:13.213594 33299 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1018 05:15:13.213528 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.213798 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1018 05:15:13.213694 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.213903 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.214096 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.214254 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:13.214300 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:13.214395 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1018 05:15:13.214584 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:13.214638 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1018 05:15:13.214766 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:13.218995 33299 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1018 05:15:13.219285 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.219491 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.219737 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.219977 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.220204 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:13.220586 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.220810 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:13.222470 33299 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1018 05:15:13.222715 33299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1018 05:15:13.222845 33299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1018 05:15:13.225872 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:13.225939 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:13.225975 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:13.226009 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:13.226041 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:13.226079 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:13.226111 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:13.226143 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:13.226175 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:13.226209 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:13.226244 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:13.226285 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:13.226325 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:13.226357 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:13.226389 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:13.226425 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:13.226456 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:13.226493 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:13.226527 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:13.226566 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:13.226599 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:13.226631 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:13.226662 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:13.226697 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:13.226732 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:13.226767 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:13.228115 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:13.228145 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:13.228186 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228236 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1018 05:15:13.228309 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228370 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228417 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228471 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228519 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228557 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1018 05:15:13.228601 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228650 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1018 05:15:13.228689 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W1018 05:15:13.228734 33299 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W1018 05:15:13.228768 33299 api.go:53] PageSize not implemented I1018 05:15:13.228790 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:13.228823 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:13.228841 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:13.228869 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:13.228889 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:13.228952 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:13.228980 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:13.229032 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:13.229073 33299 iamrole.go:89] ListRoles: { } I1018 05:15:13.229142 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:13.229188 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:13.229220 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I1018 05:15:13.229778 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.minimal.example.com ok I1018 05:15:13.229837 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1018 05:15:13.229956 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I1018 05:15:13.229847 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1018 05:15:13.230009 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:13.230000 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1018 05:15:13.230028 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I1018 05:15:13.230048 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } security-group:sg-1 ok iam-instance-profile:nodes.minimal.example.com ok I1018 05:15:13.230079 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1018 05:15:13.229981 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I1018 05:15:13.230218 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:13.230301 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1018 05:15:13.230504 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:13.230528 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:13.230564 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:13.230631 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I1018 05:15:13.230685 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1018 05:15:13.230769 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:13.230858 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I1018 05:15:13.230913 33299 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I1018 05:15:13.230883 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:13.230929 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1018 05:15:13.230977 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I1018 05:15:13.231012 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I1018 05:15:13.231053 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I1018 05:15:13.231066 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1018 05:15:13.231092 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I1018 05:15:13.231150 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:13.231231 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:13.231308 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:13.232016 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.36s) === RUN TestLifecycleMinimalOpenstack I1018 05:15:13.232270 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I1018 05:15:13.242092 33299 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:15:13.469024 33299 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1018 05:15:13.469093 33299 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1018 05:15:13.486301 33299 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1018 05:15:13.494170 33299 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1018 05:15:13.534842 33299 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1018 05:15:13.538381 33299 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1018 05:15:13.541726 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:13.541786 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:13.541920 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1018 05:15:13.547425 33299 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:15:13.738194 33299 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1018 05:15:13.738275 33299 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1018 05:15:13.760768 33299 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1018 05:15:13.764373 33299 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1018 05:15:13.772968 33299 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1018 05:15:13.778643 33299 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1018 05:15:13.782300 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:13.782531 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:13.782999 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1018 05:15:13.784708 33299 executor.go:111] Tasks: 96 done / 96 total; 0 can run ServerGroup:b0576cc7-563e-4e0e-a71b-2fc137155cc2 ok Network:72c325ed-ae40-44a3-9d46-5f2c89f4d76d ok Subnet:9a33bcd4-b794-4e6c-8140-1bf9062d0ef5 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:7d903f07-41ae-4438-9d17-2b4f6a0f08c3 ok Instance:23fcc2a3-f908-4b86-ae72-35f8336248a3 ok ServerGroup:83799dc9-3230-40f1-a74f-e8ff0de09883 ok Port:0259956f-dd29-4c45-a0e3-9aca53ade710 ok SecurityGroup:7ce7b9cb-d6e7-4496-96ba-55e524a872b8 ok Instance:6a3d9025-fe65-4f25-962c-714e5845fca7 ok Port:eedb5fae-ccf3-498c-bb90-bde19386144a ok SecurityGroup:c40e15ef-b01e-4749-8629-380fe1fd1c80 ok Port:2cebad13-5cd6-461f-bdf8-eea051f734b0 ok Instance:bc802685-26c2-4505-bf1c-54323fee7de4 ok --- PASS: TestLifecycleMinimalOpenstack (1.67s) === RUN TestLifecycleMinimalGCE I1018 05:15:14.901457 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I1018 05:15:14.903408 33299 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1018 05:15:15.134965 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:15:15.135060 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:15:15.135178 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1018 05:15:15.135215 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1018 05:15:15.135240 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:15:15.135275 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:15:15.135347 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:15:15.137147 33299 executor.go:111] Tasks: 0 done / 64 total; 41 can run W1018 05:15:15.140489 33299 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I1018 05:15:15.140549 33299 executor.go:111] Tasks: 40 done / 64 total; 18 can run I1018 05:15:15.145760 33299 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1018 05:15:15.148402 33299 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1018 05:15:15.149630 33299 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1018 05:15:15.149775 33299 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1018 05:15:15.355958 33299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1018 05:15:15.356031 33299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1018 05:15:15.356105 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1018 05:15:15.356131 33299 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1018 05:15:15.356149 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:15:15.356173 33299 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1018 05:15:15.356247 33299 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1018 05:15:15.358396 33299 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1018 05:15:15.361375 33299 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1018 05:15:15.366693 33299 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1018 05:15:15.369402 33299 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1018 05:15:15.369586 33299 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1018 05:15:15.371323 33299 gce.go:98] Scanning zones: [us-test1-a] I1018 05:15:15.371751 33299 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I1018 05:15:15.371779 33299 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:master-to-master-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1666070115 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1666070115 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.47s) === RUN TestLifecycleFloatingIPOpenstack I1018 05:15:15.373536 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I1018 05:15:15.385838 33299 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:15:15.597126 33299 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1018 05:15:15.597170 33299 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1018 05:15:15.613682 33299 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1018 05:15:15.623637 33299 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1018 05:15:15.676121 33299 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1018 05:15:15.679539 33299 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1018 05:15:15.682062 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:15.682313 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:15.682357 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1018 05:15:15.687992 33299 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1018 05:15:15.879873 33299 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1018 05:15:15.879921 33299 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1018 05:15:15.903536 33299 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1018 05:15:15.907528 33299 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1018 05:15:15.919039 33299 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1018 05:15:15.921385 33299 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1018 05:15:15.924908 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:15.925238 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1018 05:15:15.925286 33299 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1018 05:15:15.927095 33299 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:b3989b21-8612-40aa-8a3b-7b1c13f2e420 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Subnet:b3989b21-8612-40aa-8a3b-7b1c13f2e420 ok Instance:eb17431a-bbe4-4961-b030-5162e0a3258e ok SecurityGroup:25bfa609-f3ae-4940-a12b-8e4229a89a41 ok Network:b3567aad-c983-4b3a-8d76-223b5af0a8df ok Instance:8244bac4-b26d-428c-a403-1fc1c749435d ok ServerGroup:37008d15-6beb-4b6c-a9f9-63aabd01369e ok SecurityGroup:56c02ad5-c0a7-419b-a84b-6bb2f352ae78 ok Port:c4f3cb5b-70fa-46bb-9985-5981ac2d7cb6 ok ServerGroup:30510aa0-298d-4474-bdff-1414cb589296 ok Port:b256936e-c8d6-4661-93f3-0b740339eff0 ok Router:15eb9564-3ce4-47fc-a50d-22c15d57b414 ok Port:581bb2ee-caf4-4af7-a0df-aadd61d9643e ok Instance:c9f6209b-330e-4d7d-8f6f-c1f938f53cc4 ok SecurityGroup:c978349a-0dd8-4657-9e5b-871536d92467 ok Port:ed7efa95-c9a7-4ba9-95ff-c06c55889c19 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.67s) === RUN TestLifecyclePrivateCalico I1018 05:15:17.044886 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:17.045184 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:17.045242 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:17.045291 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:17.045347 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:17.045395 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:17.045426 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:17.045479 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:17.045508 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:17.045577 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I1018 05:15:17.076671 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:15:17.082473 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.082579 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.082641 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.122124 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:17.567962 33299 iamrole.go:89] ListRoles: { } I1018 05:15:17.569522 33299 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1018 05:15:17.570590 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:17.570985 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I1018 05:15:17.570965 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.571094 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1018 05:15:17.571209 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I1018 05:15:17.571308 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1018 05:15:17.571244 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.571625 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:17.571658 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I1018 05:15:17.571705 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:17.571731 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1018 05:15:17.571883 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.572048 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:17.572123 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:17.572598 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:17.572690 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.573043 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.572865 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.574563 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.575434 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:17.575669 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:17.575712 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:17.575865 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:17.575920 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:17.575964 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:17.576506 33299 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1018 05:15:17.576686 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1018 05:15:17.576721 33299 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1018 05:15:17.576644 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1018 05:15:17.576853 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1018 05:15:17.576835 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.576988 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1018 05:15:17.577016 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:17.577081 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1018 05:15:17.577093 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1018 05:15:17.577207 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:17.577314 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1018 05:15:17.577328 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:17.577393 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1018 05:15:17.577382 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I1018 05:15:17.577489 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:17.577599 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:17.577655 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:17.577701 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1018 05:15:17.577937 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1018 05:15:17.578047 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.578243 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1018 05:15:17.578359 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1018 05:15:17.578488 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1018 05:15:17.578572 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.578721 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.578841 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:17.578942 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I1018 05:15:17.578981 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:17.579085 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:17.579154 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1018 05:15:17.579260 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:17.579348 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I1018 05:15:17.579444 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:17.579427 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:17.579491 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1018 05:15:17.579608 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:17.579737 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:17.579803 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:17.579860 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:17.579935 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:17.579975 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.580061 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:17.580109 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:17.580156 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:17.580210 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:17.580262 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:17.580313 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:17.580405 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:17.580483 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:17.580644 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:17.580731 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:17.580797 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:17.580841 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:17.580551 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1018 05:15:17.582472 33299 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1018 05:15:17.582762 33299 api.go:53] PageSize not implemented I1018 05:15:17.582688 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W1018 05:15:17.582846 33299 api.go:53] PageSize not implemented I1018 05:15:17.582935 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.582993 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1018 05:15:17.583086 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.583189 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.583182 33299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I1018 05:15:17.583291 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1018 05:15:17.583334 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1018 05:15:17.583460 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1018 05:15:17.583422 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1018 05:15:17.583576 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1018 05:15:17.583646 33299 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1018 05:15:17.583706 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1018 05:15:17.583640 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.583737 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1018 05:15:17.583843 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1018 05:15:17.583954 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.584081 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.584268 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:17.584398 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:17.584528 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:17.584640 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:17.584722 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:17.584746 33299 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.584807 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.584972 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:17.585120 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1018 05:15:17.585229 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I1018 05:15:17.585374 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:17.585481 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1018 05:15:17.585592 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1018 05:15:17.585684 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.585828 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.585890 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:17.585939 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I1018 05:15:17.586080 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.586157 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.586282 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I1018 05:15:17.586410 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:17.586538 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1018 05:15:17.586625 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.586766 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.586909 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.587091 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.587309 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1018 05:15:17.587370 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:17.587438 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.587645 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.587844 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:17.587936 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.588084 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.588210 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1018 05:15:17.588365 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.588528 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.588592 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1018 05:15:17.588633 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:17.588736 33299 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1018 05:15:17.588815 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:17.588844 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:17.588887 33299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:17.589034 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:17.589123 33299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1018 05:15:17.589219 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:17.589263 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:17.589304 33299 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1018 05:15:17.589370 33299 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1018 05:15:17.589541 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:17.589700 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1018 05:15:17.589777 33299 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1018 05:15:17.589820 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:17.589850 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:17.589931 33299 dns.go:238] Pre-creating DNS records I1018 05:15:17.589971 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:17.600524 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1018 05:15:17.604226 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.604329 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.604395 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:17.644493 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.089593 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.091155 33299 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1018 05:15:18.091658 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.093293 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.093528 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.091644 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:18.093615 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.093991 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.094087 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:18.094121 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:18.094156 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.094190 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.098618 33299 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1018 05:15:18.098824 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1018 05:15:18.098776 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.098969 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1018 05:15:18.098950 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.099111 33299 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1018 05:15:18.099187 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.099432 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:18.099505 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1018 05:15:18.099770 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:18.099842 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.099989 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1018 05:15:18.100142 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:18.100195 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1018 05:15:18.100354 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.100411 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1018 05:15:18.100705 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:18.100767 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1018 05:15:18.100950 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.101112 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.101223 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] }] } I1018 05:15:18.101420 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.101482 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:18.105214 33299 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1018 05:15:18.105387 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.105498 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W1018 05:15:18.105625 33299 api.go:53] PageSize not implemented I1018 05:15:18.105630 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:18.105709 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1018 05:15:18.105672 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1018 05:15:18.105782 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } W1018 05:15:18.105828 33299 api.go:53] PageSize not implemented I1018 05:15:18.105872 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1018 05:15:18.105999 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1018 05:15:18.106041 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I1018 05:15:18.106151 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1018 05:15:18.106108 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.106189 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1018 05:15:18.106226 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.106812 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.106938 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.107084 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.107210 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.107304 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.107422 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.107536 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.109942 33299 executor.go:111] Tasks: 118 done / 127 total; 5 can run W1018 05:15:18.110065 33299 api.go:53] PageSize not implemented I1018 05:15:18.110029 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.110107 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1018 05:15:18.110142 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.110235 33299 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I1018 05:15:18.110261 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:18.110290 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W1018 05:15:18.110370 33299 api.go:53] PageSize not implemented I1018 05:15:18.110401 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1018 05:15:18.110606 33299 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1018 05:15:18.110740 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.110945 33299 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1018 05:15:18.114991 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:18.115054 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:18.115089 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:18.115120 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:18.115154 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:18.115193 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1018 05:15:18.115229 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:18.115266 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:18.115301 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:18.115335 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:18.115371 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:18.115407 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:18.115446 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:18.115484 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1018 05:15:18.115521 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:18.115559 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:18.115592 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:18.115624 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:18.115660 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:18.115692 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:18.115727 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:18.115762 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:18.115798 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:18.115833 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:18.115870 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:18.115906 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.115944 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:18.115975 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:18.116013 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:18.116046 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:18.116080 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:18.116117 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:18.116152 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:18.116191 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:18.117912 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:18.117940 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:18.117979 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118041 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1018 05:15:18.118116 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118172 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118220 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118291 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118335 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118380 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1018 05:15:18.118439 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118491 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:18.118519 33299 natgateway.go:117] DescribeNatGateways: { } I1018 05:15:18.118554 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1018 05:15:18.118593 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W1018 05:15:18.118646 33299 api.go:53] PageSize not implemented I1018 05:15:18.118664 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1018 05:15:18.118716 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:18.118744 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:18.118766 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:18.118790 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:18.118807 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:18.118857 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.118884 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.118927 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:18.118971 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.119018 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:18.119064 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:18.119107 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1018 05:15:18.119142 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:18.119177 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I1018 05:15:18.119816 33299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } I1018 05:15:18.119838 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1018 05:15:18.119886 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:18.119924 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I1018 05:15:18.119947 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:bastion.privatecalico.example.com ok I1018 05:15:18.119965 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I1018 05:15:18.119995 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1018 05:15:18.120002 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I1018 05:15:18.120065 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok W1018 05:15:18.120095 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.120076 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } nat-gateway:nat-1 ok iam-instance-profile:nodes.privatecalico.example.com ok I1018 05:15:18.119908 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I1018 05:15:18.120128 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:18.120160 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1018 05:15:18.120200 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I1018 05:15:18.120257 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok volume:vol-1 ok I1018 05:15:18.120300 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:18.120402 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1018 05:15:18.120438 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I1018 05:15:18.120513 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.120570 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1018 05:15:18.120839 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.120867 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1018 05:15:18.120913 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.120995 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1018 05:15:18.121148 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-1 ok I1018 05:15:18.121194 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:18.121334 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I1018 05:15:18.121337 33299 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1018 05:15:18.121432 33299 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I1018 05:15:18.121442 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:18.121456 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1018 05:15:18.121525 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:18.121622 33299 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I1018 05:15:18.121697 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I1018 05:15:18.121762 33299 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I1018 05:15:18.121780 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1018 05:15:18.121814 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I1018 05:15:18.121655 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 } ] } W1018 05:15:18.121849 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.121843 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I1018 05:15:18.121885 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I1018 05:15:18.121881 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1018 05:15:18.121900 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1018 05:15:18.121926 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I1018 05:15:18.121934 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:18.122013 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1018 05:15:18.122133 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.122160 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1018 05:15:18.122204 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1018 05:15:18.122325 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:18.122410 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1018 05:15:18.122506 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:18.122605 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:18.122900 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.08s) === RUN TestLifecyclePrivateKopeio I1018 05:15:18.123189 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:18.123431 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:18.123479 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:18.123524 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:18.123571 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:18.123610 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:18.123638 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:18.123673 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:18.123708 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:18.123772 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I1018 05:15:18.154826 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:18.160114 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.160222 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.160287 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.200318 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.317918 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.320271 33299 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1018 05:15:18.320592 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:18.320644 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I1018 05:15:18.320774 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1018 05:15:18.320746 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.320969 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.321035 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I1018 05:15:18.321153 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.321203 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] } I1018 05:15:18.321414 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.321419 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.321482 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1018 05:15:18.321513 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.321632 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1018 05:15:18.321726 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:18.321831 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.322034 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:18.322193 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.322309 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:18.322344 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:18.322374 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:18.322534 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:18.322589 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:18.322645 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.322692 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:18.322748 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.323511 33299 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1018 05:15:18.323651 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1018 05:15:18.324606 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1018 05:15:18.323725 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1018 05:15:18.324819 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:18.324914 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1018 05:15:18.324978 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1018 05:15:18.325295 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1018 05:15:18.325346 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1018 05:15:18.325525 33299 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1018 05:15:18.325569 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1018 05:15:18.325596 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1018 05:15:18.325710 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1018 05:15:18.325595 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1018 05:15:18.325912 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1018 05:15:18.326120 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.326481 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.326654 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:18.326744 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.326923 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1018 05:15:18.327165 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1018 05:15:18.327306 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.327754 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.328129 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1018 05:15:18.329087 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I1018 05:15:18.329256 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.329376 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:18.329436 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:18.329491 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.330408 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:18.330472 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.330611 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I1018 05:15:18.330751 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:18.330743 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.330561 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1018 05:15:18.330992 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.331037 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:18.331232 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:18.331190 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:18.331313 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.331441 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.331562 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.331532 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:18.331649 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.331785 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.331896 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.332005 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:18.332067 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.332202 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1018 05:15:18.332296 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:18.332417 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:18.332516 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I1018 05:15:18.332659 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:18.332940 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1018 05:15:18.333037 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1018 05:15:18.333167 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1018 05:15:18.333218 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1018 05:15:18.333269 33299 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1018 05:15:18.333342 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:18.333390 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:18.333434 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:18.333478 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:18.333519 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:18.333561 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:18.333599 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.333646 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:18.333689 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:18.333730 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I1018 05:15:18.333847 33299 executor.go:111] Tasks: 78 done / 126 total; 39 can run W1018 05:15:18.333977 33299 api.go:53] PageSize not implemented I1018 05:15:18.333964 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1018 05:15:18.334062 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.334156 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:18.334217 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } W1018 05:15:18.334114 33299 api.go:53] PageSize not implemented I1018 05:15:18.334265 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1018 05:15:18.334345 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I1018 05:15:18.334447 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1018 05:15:18.334576 33299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1018 05:15:18.334591 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1018 05:15:18.334675 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1018 05:15:18.334696 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1018 05:15:18.334724 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1018 05:15:18.334809 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1018 05:15:18.334861 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:18.334912 33299 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1018 05:15:18.334985 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.334911 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.335048 33299 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1018 05:15:18.335114 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:18.335063 33299 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1018 05:15:18.335175 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1018 05:15:18.335214 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.335241 33299 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1018 05:15:18.335311 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1018 05:15:18.335289 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1018 05:15:18.335375 33299 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1018 05:15:18.335440 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:18.335416 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1018 05:15:18.335504 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1018 05:15:18.335627 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.336125 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.336347 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I1018 05:15:18.336454 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.336589 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.336959 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1018 05:15:18.337218 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I1018 05:15:18.337381 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.337484 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:18.337598 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.337672 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I1018 05:15:18.337832 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.338091 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.338387 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:18.338522 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" }] }] } I1018 05:15:18.338658 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:18.338768 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.338902 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1018 05:15:18.338946 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I1018 05:15:18.339065 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.339213 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.339339 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1018 05:15:18.339468 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1018 05:15:18.339607 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.339679 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.339803 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I1018 05:15:18.339843 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.339979 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:18.340108 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.340239 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.340316 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.340500 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:18.340542 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.340675 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.340753 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1018 05:15:18.340890 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.340950 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.340992 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I1018 05:15:18.341029 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.341167 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I1018 05:15:18.341252 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I1018 05:15:18.343465 33299 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1018 05:15:18.343554 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.343722 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I1018 05:15:18.343730 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.343791 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.343960 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-1" } I1018 05:15:18.343963 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.344095 33299 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1018 05:15:18.344275 33299 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1018 05:15:18.344328 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.344363 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.344418 33299 dns.go:238] Pre-creating DNS records I1018 05:15:18.344444 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.350952 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:18.354418 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.354511 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.354578 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.393651 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.511062 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.513330 33299 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1018 05:15:18.513641 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.513884 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.514064 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:18.514033 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:18.514103 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:18.514203 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.514254 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.514332 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.514419 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.514588 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.516782 33299 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1018 05:15:18.516963 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1018 05:15:18.517014 33299 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1018 05:15:18.516929 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.517156 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.517323 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1018 05:15:18.517316 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1018 05:15:18.517562 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] }] } I1018 05:15:18.518994 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.519320 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:18.519400 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1018 05:15:18.519630 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.519823 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1018 05:15:18.520089 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.520334 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.520491 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1018 05:15:18.520754 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.520989 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1018 05:15:18.521067 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.522629 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.523411 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1018 05:15:18.523573 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.523610 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:18.523639 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.523670 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:18.523700 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:18.524377 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:18.524538 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:18.527308 33299 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1018 05:15:18.527602 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1018 05:15:18.527795 33299 api.go:53] PageSize not implemented I1018 05:15:18.527751 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W1018 05:15:18.527837 33299 api.go:53] PageSize not implemented I1018 05:15:18.527887 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.527872 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.528039 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1018 05:15:18.528025 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.528104 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1018 05:15:18.528182 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.528191 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.528291 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.528326 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1018 05:15:18.528289 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1018 05:15:18.528505 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.528687 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.528835 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1018 05:15:18.528954 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1018 05:15:18.529094 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.529300 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.529415 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.529512 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:18.531685 33299 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1018 05:15:18.531782 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.531863 33299 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" W1018 05:15:18.531895 33299 api.go:53] PageSize not implemented I1018 05:15:18.531899 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.531936 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.532047 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W1018 05:15:18.532110 33299 api.go:53] PageSize not implemented I1018 05:15:18.532184 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.532367 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.532477 33299 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1018 05:15:18.532628 33299 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1018 05:15:18.537204 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:18.537313 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:18.537387 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:18.537459 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:18.537526 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:18.537599 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:18.537666 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:18.537733 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:18.537798 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:18.537873 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:18.537937 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:18.538017 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:18.538089 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1018 05:15:18.538160 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:18.538203 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:18.538238 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:18.538302 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:18.538343 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:18.538384 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1018 05:15:18.538420 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:18.538458 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:18.538495 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:18.538531 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:18.538567 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:18.538603 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.538641 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:18.538675 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:18.538714 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1018 05:15:18.538755 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1018 05:15:18.538795 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:18.538829 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:18.538862 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:18.538901 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:18.538939 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:18.538976 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:18.540862 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:18.540891 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:18.540933 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.540999 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1018 05:15:18.541081 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541143 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541191 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541242 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541284 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541336 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1018 05:15:18.541405 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541472 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:18.541506 33299 natgateway.go:117] DescribeNatGateways: { } I1018 05:15:18.541547 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1018 05:15:18.541588 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W1018 05:15:18.541679 33299 api.go:53] PageSize not implemented I1018 05:15:18.541698 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1018 05:15:18.541752 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:18.541779 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:18.541798 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:18.541821 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:18.541839 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:18.541889 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.541918 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.541967 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:18.542006 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.542050 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:18.542097 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:18.542147 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1","rtb-3"] } I1018 05:15:18.542190 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1018 05:15:18.542222 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1018 05:15:18.542883 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I1018 05:15:18.542922 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:bastion.privatekopeio.example.com ok I1018 05:15:18.542944 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I1018 05:15:18.543006 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I1018 05:15:18.543046 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I1018 05:15:18.543037 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } keypair:key-1 ok autoscaling-group:nodes.privatekopeio.example.com ok I1018 05:15:18.543118 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:18.543131 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1018 05:15:18.543180 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I1018 05:15:18.543239 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1018 05:15:18.543289 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I1018 05:15:18.543079 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:masters.privatekopeio.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I1018 05:15:18.543184 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1018 05:15:18.543400 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.543425 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-3 ok I1018 05:15:18.543478 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:18.543544 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1018 05:15:18.543578 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:18.543634 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1018 05:15:18.543746 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.543767 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:18.543798 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:18.543854 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1018 05:15:18.543934 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.543954 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1018 05:15:18.543985 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:18.544109 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I1018 05:15:18.544164 33299 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1018 05:15:18.544120 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1018 05:15:18.544180 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1018 05:15:18.544212 33299 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I1018 05:15:18.544207 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:18.544283 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.544259 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I1018 05:15:18.544345 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1018 05:15:18.544431 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:18.544448 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1018 05:15:18.544483 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I1018 05:15:18.544433 33299 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com subnet:subnet-3 ok I1018 05:15:18.544530 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I1018 05:15:18.544535 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1018 05:15:18.544580 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I1018 05:15:18.544583 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.544627 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I1018 05:15:18.544676 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1018 05:15:18.544689 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1018 05:15:18.544629 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I1018 05:15:18.544717 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } W1018 05:15:18.544728 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-role:nodes.privatekopeio.example.com ok I1018 05:15:18.544743 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:18.544818 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:18.544870 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I1018 05:15:18.544915 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1018 05:15:18.545025 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:18.545105 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:18.545374 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.42s) === RUN TestLifecycleIPv6 I1018 05:15:18.545631 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:18.545907 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:18.545955 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:18.545998 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:18.546045 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:18.546088 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:18.546117 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:18.546153 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:18.546181 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:18.546241 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I1018 05:15:18.571638 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:18.575090 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.575182 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.613752 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.786141 33299 iamrole.go:89] ListRoles: { } I1018 05:15:18.788322 33299 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1018 05:15:18.788738 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.788973 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:18.789128 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:18.789191 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:18.789198 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I1018 05:15:18.789265 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.789329 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1018 05:15:18.789445 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1018 05:15:18.789399 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:18.789521 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1018 05:15:18.789569 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.789814 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.789955 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:18.790010 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.790176 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.790250 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.790512 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.790716 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:18.790758 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:18.790792 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:18.790837 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.790873 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:18.791010 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:18.791101 33299 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1018 05:15:18.791198 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.791333 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1018 05:15:18.791320 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I1018 05:15:18.791390 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1018 05:15:18.791416 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.791463 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1018 05:15:18.791483 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:18.791590 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:18.791615 33299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1018 05:15:18.791641 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1018 05:15:18.791659 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.791749 33299 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1018 05:15:18.791751 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:18.791828 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1018 05:15:18.791880 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:18.792020 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:18.792066 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I1018 05:15:18.792153 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:18.792155 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1018 05:15:18.792197 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] } I1018 05:15:18.792294 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.792395 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:18.792457 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:18.792496 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1018 05:15:18.792614 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.792706 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:18.792784 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1018 05:15:18.792903 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:18.792993 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.793124 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:18.793180 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:18.793227 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:18.793307 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:18.793348 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:18.793386 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.793460 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:18.793765 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1018 05:15:18.796292 33299 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1018 05:15:18.796428 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.796579 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.796618 33299 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.796705 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:18.796882 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1018 05:15:18.797041 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1018 05:15:18.797379 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I1018 05:15:18.797595 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.797856 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.797979 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.798054 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I1018 05:15:18.798282 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1018 05:15:18.798394 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.798543 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.798779 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.798926 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:18.799045 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1018 05:15:18.799122 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1018 05:15:18.799159 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I1018 05:15:18.799295 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1018 05:15:18.799350 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1018 05:15:18.799490 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.799560 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.799634 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.799769 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" }] }] } I1018 05:15:18.799900 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I1018 05:15:18.800034 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.800145 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.800243 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.800443 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.800593 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1018 05:15:18.801419 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.801714 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.801803 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1018 05:15:18.802032 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.802205 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:18.802270 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:18.802390 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:18.802796 33299 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1018 05:15:18.803097 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.803303 33299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.803693 33299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1018 05:15:18.803765 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.803796 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:18.804035 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:18.804542 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:18.804989 33299 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I1018 05:15:18.805149 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:18.805205 33299 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1018 05:15:18.805171 33299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I1018 05:15:18.805271 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:18.805286 33299 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1018 05:15:18.805358 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:18.805475 33299 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1018 05:15:18.805661 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.805710 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:18.805877 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1018 05:15:18.805854 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.806728 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.806850 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:18.806983 33299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1018 05:15:18.807045 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:18.807079 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.807132 33299 dns.go:238] Pre-creating DNS records I1018 05:15:18.807158 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:18.813360 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:18.817640 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.817738 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:18.857127 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.025050 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.026834 33299 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1018 05:15:19.027240 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.027340 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:19.027471 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:19.027522 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:19.027602 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.027661 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.027750 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.027921 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I1018 05:15:19.028093 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I1018 05:15:19.028212 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.031344 33299 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1018 05:15:19.031822 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.032159 33299 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1018 05:15:19.032145 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.032308 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1018 05:15:19.032261 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1018 05:15:19.032419 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.032501 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1018 05:15:19.032616 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.034012 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1018 05:15:19.034317 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.034382 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1018 05:15:19.034527 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.034624 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1018 05:15:19.034756 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:19.034797 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.039899 33299 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1018 05:15:19.040202 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.040385 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.040629 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:19.040682 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:19.040771 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1018 05:15:19.040815 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1018 05:15:19.041065 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.041295 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.041492 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.041763 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.041887 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.041964 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.042053 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.042177 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.045148 33299 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1018 05:15:19.045253 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.045343 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:19.045332 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:19.045416 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:19.045435 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1018 05:15:19.045422 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.045481 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1018 05:15:19.045518 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1018 05:15:19.045594 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1018 05:15:19.045652 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1018 05:15:19.045890 33299 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1018 05:15:19.046005 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:19.046079 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.046154 33299 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1018 05:15:19.046195 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:19.046235 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1018 05:15:19.046241 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:19.046354 33299 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1018 05:15:19.046381 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:19.046408 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1018 05:15:19.046502 33299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1018 05:15:19.049589 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:19.049659 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:19.049697 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:19.049729 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:19.049762 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:19.049802 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1018 05:15:19.049844 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:19.049879 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:19.049911 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:19.049945 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:19.049982 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:19.050020 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:19.050064 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:19.050104 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:19.050140 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:19.050173 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:19.050208 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:19.050246 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:19.050281 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:19.050318 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:19.050357 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:19.050394 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:19.050431 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:19.050472 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:19.050507 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:19.050540 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:19.050573 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:19.050610 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:19.050647 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:19.050685 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:19.052265 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:19.052301 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:19.052346 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052405 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1018 05:15:19.052469 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052530 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052581 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052632 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052673 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052718 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1018 05:15:19.052772 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052822 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:19.052850 33299 natgateway.go:117] DescribeNatGateways: { } I1018 05:15:19.052892 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1018 05:15:19.052937 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W1018 05:15:19.052994 33299 api.go:53] PageSize not implemented I1018 05:15:19.053013 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:19.053038 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:19.053052 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1018 05:15:19.053091 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:19.053115 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:19.053131 33299 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I1018 05:15:19.053201 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.053230 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.053296 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:19.053334 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.053378 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:19.053430 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:19.053475 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1018 05:15:19.053517 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:19.053554 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1018 05:15:19.054116 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1018 05:15:19.054183 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok I1018 05:15:19.054127 33299 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I1018 05:15:19.054231 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I1018 05:15:19.054277 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } keypair:key-1 ok I1018 05:15:19.054309 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.054302 33299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I1018 05:15:19.054463 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1018 05:15:19.054552 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I1018 05:15:19.054341 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I1018 05:15:19.054407 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } iam-instance-profile:masters.minimal-ipv6.example.com ok route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok W1018 05:15:19.054625 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:19.054647 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:19.054687 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:19.054752 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I1018 05:15:19.054812 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1018 05:15:19.054870 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:19.054916 33299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1018 05:15:19.054992 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:19.055047 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1018 05:15:19.055078 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.055131 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W1018 05:15:19.055417 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:19.055437 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I1018 05:15:19.055527 33299 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1018 05:15:19.055530 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I1018 05:15:19.055586 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:19.055637 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1018 05:15:19.055663 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I1018 05:15:19.055758 33299 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1018 05:15:19.055776 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1018 05:15:19.055817 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I1018 05:15:19.055857 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1018 05:15:19.055885 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I1018 05:15:19.055964 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:19.056049 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:19.056134 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:19.056420 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I1018 05:15:19.056714 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:19.056954 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:19.057006 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:19.057049 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:19.057096 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:19.057136 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:19.057165 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:19.057212 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:19.057241 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:19.057304 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I1018 05:15:19.081671 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:19.084687 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.084782 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.123786 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.224558 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.226100 33299 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1018 05:15:19.226429 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:19.226509 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:19.226633 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:19.226680 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:19.226738 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.226809 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.226944 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.227294 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I1018 05:15:19.227277 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:19.227416 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1018 05:15:19.227478 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:19.227550 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I1018 05:15:19.227646 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1018 05:15:19.227607 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.227866 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.228065 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:19.228206 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:19.228998 33299 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1018 05:15:19.229110 33299 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1018 05:15:19.229221 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1018 05:15:19.229309 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1018 05:15:19.229444 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1018 05:15:19.229525 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.229654 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1018 05:15:19.229642 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:19.229725 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:19.229782 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I1018 05:15:19.229866 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:19.229928 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.229980 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:19.230055 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] } I1018 05:15:19.230043 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1018 05:15:19.230150 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.230357 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-12345678" } I1018 05:15:19.230406 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:19.230564 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1018 05:15:19.230803 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:19.230901 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:19.230955 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.230989 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:19.231053 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:19.232138 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1018 05:15:19.234473 33299 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1018 05:15:19.234615 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.234766 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:19.234815 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:19.234858 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.235052 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.235192 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1018 05:15:19.235323 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.235472 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:19.235674 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:19.235813 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1018 05:15:19.236022 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:19.236186 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.236263 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1018 05:15:19.236387 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.236521 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.236682 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.236792 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.236877 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.237080 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.237296 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1018 05:15:19.237364 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.238608 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.238925 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.239225 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1018 05:15:19.239515 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.246307 33299 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1018 05:15:19.246661 33299 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1018 05:15:19.246822 33299 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1018 05:15:19.246877 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.246915 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.246978 33299 dns.go:238] Pre-creating DNS records I1018 05:15:19.247008 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.253079 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:19.256555 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.256650 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.297231 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.395035 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.396352 33299 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1018 05:15:19.396662 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:19.396620 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.396826 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.397128 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.397254 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:19.397469 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:19.398672 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:19.398718 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:19.403573 33299 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1018 05:15:19.403732 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1018 05:15:19.404487 33299 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1018 05:15:19.404641 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1018 05:15:19.404597 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1018 05:15:19.404818 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.404872 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.405185 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:19.405262 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:19.405432 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.405620 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.410061 33299 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1018 05:15:19.410258 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.410427 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.410564 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.410735 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.410908 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.411172 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.411493 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.413777 33299 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1018 05:15:19.414060 33299 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1018 05:15:19.414225 33299 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1018 05:15:19.417585 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:19.417652 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:19.417690 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:19.417724 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:19.417763 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:19.417798 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:19.417828 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:19.417862 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:19.417897 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:19.417936 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:19.417977 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:19.418009 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:19.418039 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:19.418072 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:19.418101 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:19.418138 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:19.418171 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:19.418207 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:19.418238 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:19.418288 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:19.418323 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:19.418359 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:19.418396 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:19.419876 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:19.419906 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:19.419945 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.419997 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1018 05:15:19.420052 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420107 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420147 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420186 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420224 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420262 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1018 05:15:19.420310 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420363 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1018 05:15:19.420397 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W1018 05:15:19.420443 33299 api.go:53] PageSize not implemented I1018 05:15:19.420464 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:19.420493 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:19.420511 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:19.420536 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:19.420554 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:19.420604 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.420633 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.420677 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:19.420718 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.420760 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:19.420798 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:19.420826 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.sharedvpc.example.com ok autoscaling-config:lt-1 ok I1018 05:15:19.421240 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I1018 05:15:19.421309 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:19.421285 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1018 05:15:19.421378 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I1018 05:15:19.421371 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.421423 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1018 05:15:19.421472 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I1018 05:15:19.421392 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok I1018 05:15:19.421452 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1018 05:15:19.421700 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:19.421725 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:19.421762 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.421819 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1018 05:15:19.421930 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:19.421946 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I1018 05:15:19.421989 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:19.422083 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I1018 05:15:19.422136 33299 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I1018 05:15:19.422117 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:19.422152 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1018 05:15:19.422210 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I1018 05:15:19.422253 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I1018 05:15:19.422356 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1018 05:15:19.422375 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1018 05:15:19.422414 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I1018 05:15:19.422497 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:19.422812 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I1018 05:15:19.423113 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:19.423351 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:19.423405 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:19.423449 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:19.423501 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:19.423542 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:19.423572 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:19.423611 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:19.423639 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:19.423700 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I1018 05:15:19.457152 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:19.465378 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.465490 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.505633 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.705016 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.707249 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1018 05:15:19.707415 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:19.707632 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.707749 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I1018 05:15:19.707852 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I1018 05:15:19.707806 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.707926 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1018 05:15:19.708025 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:19.708182 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:19.708074 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1018 05:15:19.708295 33299 api.go:53] PageSize not implemented I1018 05:15:19.708358 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:19.708428 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1018 05:15:19.708581 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I1018 05:15:19.708618 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:19.708951 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:19.709030 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.709243 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1018 05:15:19.709407 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.709616 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:19.709727 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:19.709784 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:19.709831 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:19.709912 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.709976 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:19.710249 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:19.710551 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:15:19.710737 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:19.710784 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:19.710895 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:19.710839 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.710965 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.711065 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1018 05:15:19.711078 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1018 05:15:19.711183 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1018 05:15:19.711120 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.711050 33299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1018 05:15:19.711288 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:19.711357 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:19.711392 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1018 05:15:19.711436 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:19.711569 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:19.711563 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1018 05:15:19.711674 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1018 05:15:19.711558 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1018 05:15:19.711620 33299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1018 05:15:19.711786 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.711988 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:19.711984 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1018 05:15:19.712066 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.712282 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I1018 05:15:19.712331 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:19.712582 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:19.712790 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:19.712907 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:19.713015 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.713095 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.713189 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.712886 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1018 05:15:19.713259 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.713330 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.713415 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I1018 05:15:19.713536 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I1018 05:15:19.713638 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:19.713690 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.713733 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:19.713840 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.713889 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:19.714019 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:19.714071 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:19.714113 33299 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I1018 05:15:19.714160 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.714256 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:19.714306 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:19.714347 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:19.714396 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:19.714453 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:19.714494 33299 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I1018 05:15:19.714539 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:19.714593 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:19.714630 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1018 05:15:19.714691 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:19.716382 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:15:19.716552 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.716616 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:19.716693 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:19.716657 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.716776 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.716851 33299 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-3" }], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I1018 05:15:19.716960 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.717169 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.717057 33299 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1018 05:15:19.717306 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:19.717351 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:19.717331 33299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1018 05:15:19.717436 33299 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:19.717469 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1018 05:15:19.717596 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" } ] }] } I1018 05:15:19.717819 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1018 05:15:19.717921 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.718028 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:19.718086 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I1018 05:15:19.718127 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.718193 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.718348 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I1018 05:15:19.718529 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I1018 05:15:19.718641 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.718758 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1018 05:15:19.718926 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1018 05:15:19.719034 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1018 05:15:19.719195 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1018 05:15:19.719302 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1018 05:15:19.719358 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.719417 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" } ] }] } I1018 05:15:19.719572 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" } ] }] } I1018 05:15:19.719722 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I1018 05:15:19.719810 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.719961 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.720113 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I1018 05:15:19.720203 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I1018 05:15:19.720295 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I1018 05:15:19.720338 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:19.720377 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1018 05:15:19.720413 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.720572 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.720718 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I1018 05:15:19.720867 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.720944 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1018 05:15:19.721099 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1018 05:15:19.721189 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.721335 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1018 05:15:19.721491 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1018 05:15:19.721526 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I1018 05:15:19.721681 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1018 05:15:19.721822 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I1018 05:15:19.721923 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1018 05:15:19.721961 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.726226 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:15:19.726353 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:19.726593 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:19.726807 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:15:19.727871 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:15:19.727936 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.727980 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.728046 33299 dns.go:238] Pre-creating DNS records I1018 05:15:19.728074 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:19.736427 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:19.741581 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.741772 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.781480 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:19.973126 33299 iamrole.go:89] ListRoles: { } I1018 05:15:19.975704 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1018 05:15:19.975926 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1018 05:15:19.976157 33299 api.go:53] PageSize not implemented I1018 05:15:19.976190 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:19.976325 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.976491 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:19.976564 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:19.976629 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:19.976828 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:19.976987 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1018 05:15:19.977146 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:19.977184 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:19.977246 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.977545 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.979332 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:15:19.979489 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1018 05:15:19.979708 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:19.979685 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.979810 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:19.979902 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:19.979800 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.979910 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:19.980069 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:19.980054 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:19.980143 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:19.980150 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.980402 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:19.980469 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:19.980678 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:19.980747 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.980870 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.980984 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:19.981095 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:19.981207 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.981288 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.981358 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:19.981831 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:19.981945 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:19.981994 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:19.982109 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:19.985859 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:15:19.986169 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.986270 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:19.986389 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:19.986413 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:19.986467 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:19.986512 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:19.986474 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.986567 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:19.986613 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:19.986645 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:19.986674 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.986809 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.986918 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.987008 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.987251 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.987345 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:19.987562 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:19.987776 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.987949 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:19.989500 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:15:19.989682 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:19.989781 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:19.989806 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:19.989832 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:19.989928 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:15:19.990135 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I1018 05:15:19.996465 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.000168 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.012058 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:20.016240 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.016336 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.056489 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.249729 33299 iamrole.go:89] ListRoles: { } I1018 05:15:20.252038 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1018 05:15:20.252395 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:20.252551 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W1018 05:15:20.252841 33299 api.go:53] PageSize not implemented I1018 05:15:20.252789 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.252863 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:20.252995 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.253199 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:20.253252 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:20.253333 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.253613 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:20.253784 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.254014 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:20.254189 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.254890 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:15:20.255052 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:20.255041 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.255275 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:20.255259 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:20.255193 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:20.255342 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.255420 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:20.255488 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.255414 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.255658 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:20.255709 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:20.255744 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:20.255812 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:20.255997 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:20.256042 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.256193 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:20.256239 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.256376 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.256470 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.256556 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:20.256701 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:20.256756 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.256900 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.257065 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.257206 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.260267 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:15:20.260595 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.260673 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:20.260783 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:20.260815 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.260886 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.260804 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.260927 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.260971 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.261013 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.261045 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.261088 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.261224 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I1018 05:15:20.261264 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.261295 33299 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I1018 05:15:20.261276 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.261326 33299 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I1018 05:15:20.261358 33299 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1018 05:15:20.261434 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.261447 33299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1018 05:15:20.261516 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.261550 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.261525 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.261660 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.261794 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.261962 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.262118 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.262194 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.266749 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:15:20.267209 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:20.267378 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:20.267423 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:20.267495 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.267724 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:15:20.268709 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1018 05:15:20.268768 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.268806 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:20.268891 33299 dns.go:238] Pre-creating DNS records I1018 05:15:20.278236 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:20.282605 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.282704 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.322032 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.512596 33299 iamrole.go:89] ListRoles: { } I1018 05:15:20.515758 33299 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1018 05:15:20.515981 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1018 05:15:20.516118 33299 api.go:53] PageSize not implemented I1018 05:15:20.516054 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.516222 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:20.516261 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:20.516325 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.516372 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:20.516487 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:20.516671 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.516877 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.517019 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.517153 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:20.517259 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:20.518383 33299 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1018 05:15:20.518510 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:20.518676 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:20.518779 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.518726 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:20.518853 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:20.518943 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:20.518927 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:20.518906 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.519026 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.519099 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.519268 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:20.519317 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.519413 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.519498 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:20.519656 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:20.519705 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.519852 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.520018 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:20.520062 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:20.520240 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:20.520334 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.520480 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.520641 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:20.520685 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1018 05:15:20.520822 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.523706 33299 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1018 05:15:20.523852 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.524043 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.524133 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.524146 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:20.524238 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:20.524261 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.524306 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.524342 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.524380 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.524353 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.524417 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.524449 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.524539 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.524675 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.524917 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.525210 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.525432 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.525553 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.525715 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.528384 33299 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1018 05:15:20.528495 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:20.528592 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:20.528621 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:20.528648 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.528742 33299 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1018 05:15:20.528895 33299 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I1018 05:15:20.532745 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.536439 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.548587 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:20.552853 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.552955 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.591749 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.785223 33299 iamrole.go:89] ListRoles: { } I1018 05:15:20.787546 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1018 05:15:20.787845 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:20.788065 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1018 05:15:20.787955 33299 api.go:53] PageSize not implemented I1018 05:15:20.788224 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:20.788368 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:20.788476 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.788695 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:20.788869 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.789080 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:20.789251 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:20.789411 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.789571 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:20.789619 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:20.789714 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.790703 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:20.790901 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:20.790845 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.791003 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:20.791108 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:20.791050 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:20.791101 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.791241 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.791221 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.791338 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:20.791353 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.791478 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.791641 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.791733 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.791814 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.791958 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:20.792002 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:20.792157 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:20.792222 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:20.792470 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:20.792554 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:20.792691 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:20.792745 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.792893 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:20.792939 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:20.793067 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:20.796233 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:20.796380 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.796471 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.796604 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.796646 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:20.796758 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:20.796779 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.796807 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.796846 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.796978 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.797023 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:20.796996 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.797061 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:20.797168 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.797194 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:20.797550 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.797698 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.798005 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:20.798174 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:20.798453 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.800055 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.800133 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.800313 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:20.800389 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:20.800467 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:20.800492 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:20.800515 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:20.800592 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:20.800705 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:15:20.800753 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:20.800792 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:20.800846 33299 dns.go:238] Pre-creating DNS records I1018 05:15:20.808821 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:20.813142 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.813246 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:20.853125 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.053322 33299 iamrole.go:89] ListRoles: { } I1018 05:15:21.055298 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1018 05:15:21.055549 33299 api.go:53] PageSize not implemented I1018 05:15:21.055522 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:21.055735 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:21.056008 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.056151 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:21.056211 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:21.058304 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:21.058509 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.058815 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.059278 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:21.059543 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.060215 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:21.060268 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:21.060377 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.061762 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:21.062106 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:21.062264 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.062266 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.062383 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:21.062486 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.062572 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:21.062555 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.062777 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:21.062639 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.063621 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:21.063902 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:21.063979 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.064074 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1018 05:15:21.064235 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:21.064291 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.065281 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:21.065492 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:21.065550 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.065701 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:21.065746 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.065889 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.066027 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.066191 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:21.066237 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:21.066381 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:21.069743 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:21.069892 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.070100 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.070162 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:21.070234 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:21.070253 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.070297 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.070258 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.070337 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.070461 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.070533 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.070473 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.070664 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.070663 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.070703 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.070775 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.070945 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.071713 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.072087 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.072291 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.073699 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:21.073786 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:21.073872 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:21.073900 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:21.073926 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.074083 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:21.074226 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I1018 05:15:21.079704 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.083369 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.083467 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.093043 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:21.097238 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.097335 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.097396 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.136077 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.326037 33299 iamrole.go:89] ListRoles: { } I1018 05:15:21.327925 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1018 05:15:21.328414 33299 api.go:53] PageSize not implemented I1018 05:15:21.328315 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.328615 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:21.328645 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:21.328775 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:21.328949 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.329119 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:21.329227 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.330377 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:21.331130 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1018 05:15:21.331440 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:21.331563 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:21.331839 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:21.332009 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.334376 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:21.334518 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.334628 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.335738 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:21.335714 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:21.335822 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:21.335707 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I1018 05:15:21.335856 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.335994 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:21.335935 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:21.336081 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.336070 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.336235 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.336381 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:21.336426 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:21.336590 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:21.336658 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.336912 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.337166 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:21.337237 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.337395 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.337568 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.337698 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:21.337922 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:21.338171 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:21.338257 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:21.341670 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:21.341815 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.341980 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:21.342105 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:21.342052 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.342135 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.342201 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.342245 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.342288 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.342244 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.342337 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.342388 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.342439 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.342711 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.342877 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.342976 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.343196 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.343297 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.343481 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.343555 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.345179 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:21.345291 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:21.345389 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:21.345416 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:21.345442 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.345572 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:21.345717 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:15:21.345763 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.345798 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:21.345857 33299 dns.go:238] Pre-creating DNS records I1018 05:15:21.354437 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:21.359208 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.359308 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.359371 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.398472 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.592794 33299 iamrole.go:89] ListRoles: { } I1018 05:15:21.595144 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1018 05:15:21.595364 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W1018 05:15:21.595650 33299 api.go:53] PageSize not implemented I1018 05:15:21.595600 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:21.595671 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1018 05:15:21.595853 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.596000 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:21.596228 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:21.596384 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1018 05:15:21.596570 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:21.596678 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:21.596793 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.596901 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:21.596929 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:21.596984 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.598622 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:21.598748 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:21.598789 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.598985 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.599038 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:21.599132 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.599198 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:21.599089 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:21.599233 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:21.599326 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.599359 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:21.599485 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.599678 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:21.599724 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.599859 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.599949 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.600026 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:21.600168 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:21.600230 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:21.600440 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:21.600544 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.600757 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:21.600821 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.600943 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.601124 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:21.601167 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.604175 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:21.604323 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.604514 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.604542 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:21.604625 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:21.604649 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.604695 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.604733 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.604773 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.604810 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.604793 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.604843 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.605032 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.605157 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.605494 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.605814 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.605933 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.606001 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.606107 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.606229 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.609223 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:21.609308 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:21.609393 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:21.609421 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:21.609447 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.609530 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:21.609672 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I1018 05:15:21.612915 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.616499 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.616588 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.616646 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.626245 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:21.630425 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.630525 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.630586 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.630638 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.670316 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.862176 33299 iamrole.go:89] ListRoles: { } I1018 05:15:21.864440 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1018 05:15:21.864653 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:21.864794 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:21.864891 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:21.865041 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W1018 05:15:21.865232 33299 api.go:53] PageSize not implemented I1018 05:15:21.865301 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:21.865408 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:21.865467 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.865657 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.865833 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:21.865877 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:21.865958 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.866032 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.866256 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1018 05:15:21.867001 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:21.867140 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.867321 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:21.867380 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.867432 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:21.867477 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.867480 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:21.867560 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.867590 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:21.867550 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:21.867622 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:21.867714 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.867783 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:21.868558 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.868809 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.869046 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.869249 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:21.869514 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:21.869578 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.869714 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.869858 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:21.870047 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:21.870152 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:21.870308 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:21.870508 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:21.870630 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:21.876782 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:21.876938 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:21.877034 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:21.877056 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.877103 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.877028 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.877143 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.877184 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:21.877168 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.877232 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:21.877289 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:21.877423 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:21.877574 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.877787 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.878037 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.878378 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.878656 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.878997 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.879148 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:21.879426 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.880799 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:21.880933 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:21.881051 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:21.881086 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:21.881120 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:21.881220 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:21.881337 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:15:21.881408 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:21.881474 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:21.881561 33299 dns.go:238] Pre-creating DNS records I1018 05:15:21.889796 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1018 05:15:21.894200 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.894310 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.894392 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.894464 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:21.933706 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.110392 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.112741 33299 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1018 05:15:22.112982 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1018 05:15:22.113156 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:22.113201 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W1018 05:15:22.113285 33299 api.go:53] PageSize not implemented I1018 05:15:22.113354 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.113498 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1018 05:15:22.113652 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1018 05:15:22.113822 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.114026 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.114167 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:22.114209 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:22.114275 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.114326 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.114457 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1018 05:15:22.115442 33299 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1018 05:15:22.115651 33299 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:22.115589 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.115721 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:22.115750 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.115847 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1018 05:15:22.115931 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:22.115908 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.115996 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.116043 33299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1018 05:15:22.116139 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:22.116126 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.116218 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1018 05:15:22.116383 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.116434 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1018 05:15:22.116577 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:22.116637 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.116784 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:22.116824 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1018 05:15:22.116981 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.117077 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.117281 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.117439 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:22.117485 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1018 05:15:22.117524 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.117611 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.120931 33299 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1018 05:15:22.121191 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.121301 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:22.121389 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:22.121408 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:22.121319 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:22.121455 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:22.121588 33299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:22.121660 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1018 05:15:22.121698 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1018 05:15:22.121529 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.121732 33299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:22.121809 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.122086 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.122317 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.122447 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.122542 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:22.122860 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.123031 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.123206 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:22.125765 33299 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1018 05:15:22.125854 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:22.125938 33299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1018 05:15:22.125963 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1018 05:15:22.125989 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:22.126068 33299 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1018 05:15:22.126174 33299 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1018 05:15:22.126496 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:22.126558 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:22.126595 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:22.126630 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:22.126672 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:22.126715 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:22.126749 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:22.126784 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:22.126821 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:22.126863 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:22.126909 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:22.126951 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:22.127003 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:22.127039 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:22.127081 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:22.127115 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:22.127153 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:22.127192 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:22.127229 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:22.127267 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:22.127308 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:22.127343 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:22.127382 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1018 05:15:22.127425 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:22.127460 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:22.127494 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:22.127535 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:22.127576 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:22.127617 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:22.130322 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:22.130359 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:22.130396 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130457 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1018 05:15:22.130522 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130583 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130633 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130695 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130737 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130783 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1018 05:15:22.130843 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130916 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1018 05:15:22.130958 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W1018 05:15:22.131014 33299 api.go:53] PageSize not implemented I1018 05:15:22.131035 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:22.131061 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:22.131075 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1018 05:15:22.131115 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:22.131139 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:22.131156 33299 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I1018 05:15:22.131225 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.131254 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.131298 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:22.131332 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.131373 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:22.131423 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:22.131469 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1018 05:15:22.132021 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:22.132082 33299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok I1018 05:15:22.132139 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1018 05:15:22.132228 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I1018 05:15:22.132203 33299 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1018 05:15:22.132096 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W1018 05:15:22.132321 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.132296 33299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I1018 05:15:22.132339 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1018 05:15:22.132214 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I1018 05:15:22.132336 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1018 05:15:22.132378 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:nodes.complex.example.com ok route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok volume:vol-2 ok I1018 05:15:22.132436 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1018 05:15:22.132474 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } iam-instance-profile:masters.complex.example.com ok subnet:subnet-2 ok I1018 05:15:22.132551 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:22.132605 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.132698 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:22.132734 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.132798 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 } ] } W1018 05:15:22.133149 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.133171 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:22.133205 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok autoscaling-config:lt-1 ok I1018 05:15:22.133278 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:22.133339 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I1018 05:15:22.133423 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1018 05:15:22.133478 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1018 05:15:22.133456 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I1018 05:15:22.133541 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I1018 05:15:22.133562 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1018 05:15:22.133599 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I1018 05:15:22.133648 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I1018 05:15:22.133737 33299 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I1018 05:15:22.133753 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1018 05:15:22.133788 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I1018 05:15:22.133869 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:22.133960 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:22.134041 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:22.134337 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.71s) === RUN TestLifecycleExternalLB I1018 05:15:22.134615 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:22.134858 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:22.134909 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:22.134978 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:22.135025 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:22.135067 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:22.135096 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:22.135134 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:22.135159 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:22.135224 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I1018 05:15:22.160029 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:22.163136 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.163244 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.202726 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.304366 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.306175 33299 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1018 05:15:22.306494 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } W1018 05:15:22.306683 33299 api.go:53] PageSize not implemented I1018 05:15:22.306627 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I1018 05:15:22.306467 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:22.306870 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W1018 05:15:22.306942 33299 api.go:53] PageSize not implemented I1018 05:15:22.306823 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.306947 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.307096 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.307042 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } W1018 05:15:22.307239 33299 api.go:53] PageSize not implemented I1018 05:15:22.307213 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:22.307303 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:22.307280 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.307300 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I1018 05:15:22.307409 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:22.307498 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:22.307547 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1018 05:15:22.307614 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.307726 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.307589 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.307875 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:22.308034 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:22.308101 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.308176 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.308257 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.308389 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:22.308514 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:22.308459 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.308564 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.308762 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:22.308802 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:22.308832 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:22.308874 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.310837 33299 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1018 05:15:22.311040 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.311268 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1018 05:15:22.311531 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1018 05:15:22.311685 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1018 05:15:22.311552 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I1018 05:15:22.311357 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1018 05:15:22.312078 33299 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1018 05:15:22.312159 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1018 05:15:22.312649 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1018 05:15:22.312856 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1018 05:15:22.312991 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.313490 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.314208 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:22.314265 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.314405 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:22.314505 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:22.314621 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:22.314697 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:22.315670 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.315809 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:22.315908 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] }], VpcId: "vpc-1" } I1018 05:15:22.316057 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:22.316167 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.316082 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1018 05:15:22.316240 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:22.316318 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] } I1018 05:15:22.316417 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:22.316472 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:22.316516 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.316909 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1018 05:15:22.319104 33299 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1018 05:15:22.319427 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.319761 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.320020 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.320334 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.320484 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.320624 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:22.320681 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.320828 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.320976 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.321046 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.321128 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1018 05:15:22.321266 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" }] }] } I1018 05:15:22.321469 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.321676 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:22.321817 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.321939 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.322066 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.322200 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:22.322504 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:22.322563 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:22.322602 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I1018 05:15:22.322751 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:22.322803 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1018 05:15:22.322939 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.323033 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.324563 33299 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1018 05:15:22.324834 33299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1018 05:15:22.325064 33299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1018 05:15:22.325126 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.325166 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.325222 33299 dns.go:238] Pre-creating DNS records I1018 05:15:22.325253 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.331272 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:22.334410 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.334515 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.373932 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.477678 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.479428 33299 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1018 05:15:22.479660 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:22.479847 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W1018 05:15:22.479892 33299 api.go:53] PageSize not implemented I1018 05:15:22.479856 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W1018 05:15:22.480011 33299 api.go:53] PageSize not implemented I1018 05:15:22.480049 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:22.480115 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:22.480008 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1018 05:15:22.480214 33299 api.go:53] PageSize not implemented I1018 05:15:22.480213 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.480039 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:22.480240 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.480285 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.480444 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.480430 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.480511 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.480547 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1018 05:15:22.480636 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.480692 33299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:22.480810 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:22.480782 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.482546 33299 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1018 05:15:22.482739 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1018 05:15:22.482668 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1018 05:15:22.482859 33299 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1018 05:15:22.482856 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.482995 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.483179 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.483344 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:22.483392 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.483522 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.483615 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:22.483703 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1018 05:15:22.483867 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.487552 33299 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1018 05:15:22.487772 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.487872 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.488233 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.488436 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.488584 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:22.488728 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.488873 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.490683 33299 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1018 05:15:22.490903 33299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1018 05:15:22.491046 33299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1018 05:15:22.494230 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:22.494297 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:22.494335 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:22.494367 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:22.494403 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:22.494442 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:22.494478 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:22.494512 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:22.494546 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:22.494585 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:22.494619 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:22.494657 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:22.494700 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:22.494734 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:22.494768 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:22.494804 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:22.494838 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:22.494873 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:22.494909 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:22.494965 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:22.495003 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:22.495038 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:22.495073 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:22.495110 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:22.495149 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:22.495184 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:22.496553 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:22.496584 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:22.496626 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496680 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1018 05:15:22.496738 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496794 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496843 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496898 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496942 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.496983 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1018 05:15:22.497030 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.497081 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1018 05:15:22.497123 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W1018 05:15:22.497178 33299 api.go:53] PageSize not implemented I1018 05:15:22.497199 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:22.497228 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:22.497245 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:22.497268 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:22.497284 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:22.497360 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.497391 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.497439 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:22.497473 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.497515 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:22.497559 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:22.497592 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I1018 05:15:22.498029 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1018 05:15:22.498097 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:nodes.externallb.example.com ok I1018 05:15:22.498083 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } volume:vol-1 ok I1018 05:15:22.498143 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:22.498168 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I1018 05:15:22.498202 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } iam-instance-profile:nodes.externallb.example.com ok I1018 05:15:22.498231 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } internet-gateway:igw-2 ok I1018 05:15:22.498252 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1018 05:15:22.498350 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I1018 05:15:22.498193 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.498302 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.externallb.example.com. ok W1018 05:15:22.498442 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.498458 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I1018 05:15:22.498509 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.498566 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W1018 05:15:22.498754 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.498774 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I1018 05:15:22.498823 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:22.498922 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I1018 05:15:22.499027 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I1018 05:15:22.498938 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:22.499046 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1018 05:15:22.499091 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I1018 05:15:22.499168 33299 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I1018 05:15:22.499187 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1018 05:15:22.499226 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I1018 05:15:22.499262 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I1018 05:15:22.499323 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:22.499401 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:22.499475 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:22.499727 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I1018 05:15:22.500001 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:22.500234 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:22.500280 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:22.500319 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:22.500368 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:22.500416 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:22.500445 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:22.500480 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:22.500508 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:22.500573 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I1018 05:15:22.524678 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:22.527717 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.527816 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.567385 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.666609 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.668105 33299 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1018 05:15:22.668399 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:22.668519 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:22.668460 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I1018 05:15:22.668601 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1018 05:15:22.668556 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:22.668938 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.669057 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1018 05:15:22.669029 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:22.669029 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.669325 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.669555 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:22.669738 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:22.669808 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.669888 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.670055 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" }] }] } I1018 05:15:22.670245 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:22.670372 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:22.671303 33299 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1018 05:15:22.671580 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1018 05:15:22.671585 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1018 05:15:22.671781 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1018 05:15:22.671821 33299 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1018 05:15:22.671876 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.671815 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1018 05:15:22.672066 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I1018 05:15:22.672163 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:22.672216 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.672252 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:22.672366 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:22.672411 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.672403 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.672452 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1018 05:15:22.672678 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:22.672810 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:22.672864 33299 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" } ] } I1018 05:15:22.676636 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1018 05:15:22.681259 33299 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1018 05:15:22.681462 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.681669 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I1018 05:15:22.681889 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:22.682109 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1018 05:15:22.682267 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.682469 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.682804 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I1018 05:15:22.683013 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1018 05:15:22.683254 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.683389 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.683603 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.683886 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:22.684111 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.684335 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1018 05:15:22.684520 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.684663 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1018 05:15:22.684896 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.685022 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.686996 33299 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1018 05:15:22.687330 33299 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1018 05:15:22.687481 33299 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1018 05:15:22.687545 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.687604 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.687699 33299 dns.go:238] Pre-creating DNS records I1018 05:15:22.687748 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.693969 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:22.697084 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.697175 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.736489 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.832776 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.834035 33299 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1018 05:15:22.834341 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.834864 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:22.835975 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:22.837072 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.837846 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.837989 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:22.838045 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:22.838063 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:22.839425 33299 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1018 05:15:22.839538 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1018 05:15:22.839576 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1018 05:15:22.839694 33299 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1018 05:15:22.839742 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.839789 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1018 05:15:22.839923 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1018 05:15:22.840090 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.840148 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:22.842691 33299 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1018 05:15:22.842952 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.843072 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.843165 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.843479 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:22.846496 33299 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1018 05:15:22.846708 33299 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1018 05:15:22.846820 33299 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1018 05:15:22.850612 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:22.850681 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:22.850719 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:22.850765 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:22.850802 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:22.850862 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:22.850903 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:22.850939 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:22.850984 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:22.851028 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:22.851088 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:22.851128 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:22.851166 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:22.851200 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:22.851234 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:22.851282 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:22.851353 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:22.851413 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:22.851473 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:22.851544 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:22.851631 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:22.853529 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:22.853562 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:22.853619 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.853695 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1018 05:15:22.853765 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.853848 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.853900 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.853975 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.854025 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.854060 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1018 05:15:22.854113 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.854165 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1018 05:15:22.854203 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W1018 05:15:22.854269 33299 api.go:53] PageSize not implemented I1018 05:15:22.854295 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:22.854336 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:22.854358 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:22.854392 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:22.854414 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:22.854465 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:22.854502 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:22.854551 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:22.854595 33299 iamrole.go:89] ListRoles: { } I1018 05:15:22.854650 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:22.854689 33299 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok I1018 05:15:22.855168 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1018 05:15:22.855174 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I1018 05:15:22.855252 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.855286 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1018 05:15:22.855337 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I1018 05:15:22.855369 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I1018 05:15:22.855249 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I1018 05:15:22.855367 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:22.855462 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1018 05:15:22.855619 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.855635 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I1018 05:15:22.855672 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:22.855709 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1018 05:15:22.855782 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:22.855791 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:22.855823 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:22.855923 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I1018 05:15:22.855966 33299 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1018 05:15:22.855978 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1018 05:15:22.856014 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I1018 05:15:22.856042 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.856096 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1018 05:15:22.856105 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1018 05:15:22.856127 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I1018 05:15:22.856349 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.36s) === RUN TestLifecyclePrivateSharedSubnet I1018 05:15:22.857089 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:22.857429 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:22.857494 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:22.857547 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:22.857598 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:22.857643 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:22.857685 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:22.857748 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:22.857785 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:22.857861 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I1018 05:15:22.888583 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:22.891751 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:22.891851 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:22.891917 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:22.931250 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.041886 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.047981 33299 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1018 05:15:23.048644 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.048698 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:23.048807 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" }] } I1018 05:15:23.048735 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.049033 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I1018 05:15:23.049155 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1018 05:15:23.049060 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.049363 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:23.049350 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.049469 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1018 05:15:23.049453 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.049805 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.050062 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:23.050150 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:23.050275 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:23.050324 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:23.050373 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.050473 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.050714 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.051298 33299 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1018 05:15:23.051570 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.051766 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1018 05:15:23.051662 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1018 05:15:23.051814 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1018 05:15:23.051929 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1018 05:15:23.052097 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.052203 33299 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1018 05:15:23.052243 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1018 05:15:23.052172 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.052320 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:23.052378 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.052389 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1018 05:15:23.052435 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1018 05:15:23.052580 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.052676 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1018 05:15:23.052870 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1018 05:15:23.053108 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:23.053235 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1018 05:15:23.053393 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1018 05:15:23.053609 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.053727 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:23.053782 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.053822 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.053914 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:23.053952 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.054050 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:23.054099 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.054142 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:23.054185 33299 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I1018 05:15:23.054282 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.054435 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:23.054510 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:23.054580 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:23.054105 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.054675 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.054750 33299 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I1018 05:15:23.056994 33299 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1018 05:15:23.057159 33299 api.go:53] PageSize not implemented I1018 05:15:23.057257 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.057369 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1018 05:15:23.057510 33299 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1018 05:15:23.057594 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1018 05:15:23.057485 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W1018 05:15:23.057641 33299 api.go:53] PageSize not implemented I1018 05:15:23.057675 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1018 05:15:23.057765 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.057839 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1018 05:15:23.058031 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1018 05:15:23.058128 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1018 05:15:23.058166 33299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1018 05:15:23.058071 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.058227 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1018 05:15:23.058257 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1018 05:15:23.058324 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1018 05:15:23.058286 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1018 05:15:23.058560 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" }] }] } I1018 05:15:23.058724 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1018 05:15:23.059049 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.059207 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:23.059396 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1018 05:15:23.059559 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I1018 05:15:23.059706 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.059838 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1018 05:15:23.060006 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.060246 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.060420 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.060552 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I1018 05:15:23.060876 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.061043 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.061203 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.061327 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.061402 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.061624 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.061799 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1018 05:15:23.061895 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:23.062025 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.062180 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.062336 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.062453 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.062514 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1018 05:15:23.065041 33299 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1018 05:15:23.065369 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:23.065616 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:23.065829 33299 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1018 05:15:23.066176 33299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1018 05:15:23.066392 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.066538 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.066781 33299 dns.go:238] Pre-creating DNS records I1018 05:15:23.066907 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.075235 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:23.078592 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.078692 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.078760 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.118520 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.216384 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.218097 33299 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1018 05:15:23.218296 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.218356 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.218461 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.218601 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:23.218583 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:23.218662 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:23.218685 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:23.218755 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.225591 33299 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1018 05:15:23.225744 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1018 05:15:23.225857 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1018 05:15:23.225986 33299 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1018 05:15:23.226092 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1018 05:15:23.226052 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:23.226229 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1018 05:15:23.226494 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.226587 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1018 05:15:23.226771 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1018 05:15:23.227045 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.227116 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:23.227176 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1018 05:15:23.227362 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1018 05:15:23.227542 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1018 05:15:23.227764 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.227881 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.232043 33299 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1018 05:15:23.232186 33299 api.go:53] PageSize not implemented I1018 05:15:23.232211 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1018 05:15:23.232310 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1018 05:15:23.232371 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1018 05:15:23.232353 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W1018 05:15:23.232490 33299 api.go:53] PageSize not implemented I1018 05:15:23.232523 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1018 05:15:23.232627 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1018 05:15:23.232529 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.232670 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1018 05:15:23.232788 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.233338 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.233531 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.233676 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.235517 33299 executor.go:111] Tasks: 100 done / 107 total; 4 can run W1018 05:15:23.235626 33299 api.go:53] PageSize not implemented I1018 05:15:23.235632 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:23.235652 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1018 05:15:23.235729 33299 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I1018 05:15:23.235756 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W1018 05:15:23.235824 33299 api.go:53] PageSize not implemented I1018 05:15:23.235856 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1018 05:15:23.236065 33299 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1018 05:15:23.236310 33299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1018 05:15:23.240045 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:23.240110 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:23.240148 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:23.240192 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:23.240225 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:23.240259 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:23.240289 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:23.240322 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:23.240357 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:23.240396 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:23.240432 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1018 05:15:23.240476 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:23.240510 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:23.240543 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:23.240577 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:23.240612 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:23.240647 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:23.240681 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:23.240717 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:23.240751 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:23.240788 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:23.240822 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:23.240854 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:23.240889 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:23.240923 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:23.242588 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:23.242619 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:23.242662 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.242723 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1018 05:15:23.242802 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.242861 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.242902 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.242942 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.242983 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.243018 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1018 05:15:23.243055 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.243100 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1018 05:15:23.243137 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W1018 05:15:23.243183 33299 api.go:53] PageSize not implemented I1018 05:15:23.243200 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1018 05:15:23.243274 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:23.243301 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:23.243321 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:23.243345 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:23.243362 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:23.243413 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.243438 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.243484 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:23.243522 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.243568 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:23.243610 33299 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I1018 05:15:23.244144 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.244156 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.244248 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I1018 05:15:23.244217 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } autoscaling-config:lt-2 ok I1018 05:15:23.244212 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } load-balancer:bastion-private-shared-su-5ol32q ok W1018 05:15:23.244305 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.244322 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:23.244320 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } autoscaling-group:bastion.private-shared-subnet.example.com ok I1018 05:15:23.244404 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I1018 05:15:23.244417 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1018 05:15:23.244357 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } load-balancer:api-private-shared-subnet-n2f8ak ok volume:vol-2 ok I1018 05:15:23.244448 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.244494 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I1018 05:15:23.244487 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:23.244543 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:nodes.private-shared-subnet.example.com ok I1018 05:15:23.244596 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1018 05:15:23.244710 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.244526 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.244733 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok security-group:sg-5 ok autoscaling-config:lt-3 ok I1018 05:15:23.244763 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.244817 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1018 05:15:23.245073 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.245096 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I1018 05:15:23.245139 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:23.245226 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.245281 33299 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1018 05:15:23.245289 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.245305 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.245386 33299 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1018 05:15:23.245402 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1018 05:15:23.245382 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I1018 05:15:23.245461 33299 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } W1018 05:15:23.245478 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-role:bastions.private-shared-subnet.example.com ok I1018 05:15:23.245495 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:23.245495 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I1018 05:15:23.245539 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1018 05:15:23.245533 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.245554 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1018 05:15:23.245585 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W1018 05:15:23.245684 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.245681 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I1018 05:15:23.245698 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1018 05:15:23.245761 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1018 05:15:23.245796 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I1018 05:15:23.246066 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedIP I1018 05:15:23.246336 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:23.246585 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:23.246633 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:23.246677 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:23.246725 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:23.246766 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:23.246799 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:23.246838 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:23.246865 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:23.246942 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I1018 05:15:23.275258 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:23.278555 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.278647 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.278708 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.318709 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.426072 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.431401 33299 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1018 05:15:23.431719 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.432325 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.432384 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.432503 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] } I1018 05:15:23.432557 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:23.432605 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I1018 05:15:23.432692 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:23.432737 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:23.432576 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:23.432721 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1018 05:15:23.432913 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.433034 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" }] } I1018 05:15:23.432800 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.433202 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:23.433300 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.433534 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:23.433614 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.433726 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.433907 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.435833 33299 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1018 05:15:23.435993 33299 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1018 05:15:23.436019 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.436197 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1018 05:15:23.436275 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1018 05:15:23.436200 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:23.436340 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I1018 05:15:23.436381 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.436387 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.436558 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1018 05:15:23.436558 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.436657 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1018 05:15:23.436777 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I1018 05:15:23.436956 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:23.436908 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.437081 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.437045 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I1018 05:15:23.437159 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1018 05:15:23.437194 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1018 05:15:23.437368 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.437506 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1018 05:15:23.437672 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I1018 05:15:23.437728 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1018 05:15:23.437943 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.438117 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1018 05:15:23.438256 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.438390 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.438477 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:23.438550 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.438637 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:23.438717 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:23.438787 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.438899 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:23.438942 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.438985 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.439127 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:23.439192 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.439240 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:23.439382 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I1018 05:15:23.439049 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1018 05:15:23.439444 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:23.439545 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:23.439596 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1018 05:15:23.439703 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:23.439752 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:23.439793 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:23.439843 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.439884 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:23.439929 33299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1018 05:15:23.441662 33299 executor.go:111] Tasks: 71 done / 116 total; 36 can run W1018 05:15:23.441990 33299 api.go:53] PageSize not implemented I1018 05:15:23.441971 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.442166 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.442219 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.442320 33299 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1018 05:15:23.442264 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.442422 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.442451 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:23.442483 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1018 05:15:23.442564 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1018 05:15:23.442595 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } W1018 05:15:23.442641 33299 api.go:53] PageSize not implemented I1018 05:15:23.442661 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.442652 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" }] }] } I1018 05:15:23.442840 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1018 05:15:23.442881 33299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I1018 05:15:23.442818 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:23.442943 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1018 05:15:23.442975 33299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1018 05:15:23.443022 33299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1018 05:15:23.443042 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1018 05:15:23.443099 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" }] }] } I1018 05:15:23.443273 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1018 05:15:23.443354 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:23.444538 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.444872 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1018 05:15:23.445095 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I1018 05:15:23.445403 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.445668 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.445807 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:23.446078 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.446372 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1018 05:15:23.446527 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:23.446631 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1018 05:15:23.446896 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1018 05:15:23.447579 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.448116 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I1018 05:15:23.448167 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I1018 05:15:23.448310 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.448457 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.448557 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1018 05:15:23.448701 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.448826 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:23.448867 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.450026 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.450184 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I1018 05:15:23.450309 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.450451 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1018 05:15:23.450592 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1018 05:15:23.450693 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" }] }] } I1018 05:15:23.450828 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1018 05:15:23.450966 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.451058 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:23.451099 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.451153 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:23.451196 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.451302 33299 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I1018 05:15:23.453579 33299 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1018 05:15:23.453674 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:23.453825 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1018 05:15:23.453887 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.453996 33299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:23.454133 33299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1018 05:15:23.454233 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:23.454279 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:23.454321 33299 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1018 05:15:23.454400 33299 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1018 05:15:23.454462 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.454588 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1018 05:15:23.454659 33299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1018 05:15:23.454702 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.454737 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.454791 33299 dns.go:238] Pre-creating DNS records I1018 05:15:23.454819 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.461196 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1018 05:15:23.464533 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.464624 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.464691 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.504647 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.609043 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.611556 33299 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1018 05:15:23.611949 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.614364 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:23.612230 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1018 05:15:23.614807 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1018 05:15:23.614884 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] }] } I1018 05:15:23.615166 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.615387 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.615657 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1018 05:15:23.620408 33299 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1018 05:15:23.620565 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1018 05:15:23.620677 33299 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1018 05:15:23.620873 33299 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1018 05:15:23.620743 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1018 05:15:23.620965 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1018 05:15:23.621000 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1018 05:15:23.621181 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1018 05:15:23.621402 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.621468 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:23.621586 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1018 05:15:23.621851 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.621920 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.622068 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I1018 05:15:23.622177 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.622274 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1018 05:15:23.622430 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.622467 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.622618 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:23.622661 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.622714 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1018 05:15:23.626034 33299 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1018 05:15:23.626243 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W1018 05:15:23.626405 33299 api.go:53] PageSize not implemented I1018 05:15:23.626337 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1018 05:15:23.626473 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1018 05:15:23.626612 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1018 05:15:23.626706 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1018 05:15:23.626796 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:23.626866 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1018 05:15:23.626917 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W1018 05:15:23.627005 33299 api.go:53] PageSize not implemented I1018 05:15:23.627049 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.627212 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.627166 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.627273 33299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1018 05:15:23.627465 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.627541 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.627693 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.627793 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.627872 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.628142 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1018 05:15:23.631307 33299 executor.go:111] Tasks: 107 done / 116 total; 5 can run W1018 05:15:23.631422 33299 api.go:53] PageSize not implemented I1018 05:15:23.631414 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.631447 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1018 05:15:23.631517 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:23.631492 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W1018 05:15:23.631580 33299 api.go:53] PageSize not implemented I1018 05:15:23.631612 33299 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I1018 05:15:23.631630 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1018 05:15:23.631831 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1018 05:15:23.631944 33299 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1018 05:15:23.632008 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.632166 33299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1018 05:15:23.636054 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:23.636119 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:23.636159 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:23.636194 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:23.636227 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:23.636268 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:23.636308 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:23.636343 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:23.636380 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:23.636421 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:23.636462 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1018 05:15:23.636505 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1018 05:15:23.636542 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:23.636575 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:23.636609 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:23.636648 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:23.636681 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1018 05:15:23.636717 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:23.636754 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:23.636789 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1018 05:15:23.636824 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1018 05:15:23.636860 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1018 05:15:23.636896 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:23.636935 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:23.636969 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1018 05:15:23.637006 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:23.637038 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:23.637071 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:23.637108 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:23.637146 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:23.638763 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:23.638794 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:23.638838 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.638902 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1018 05:15:23.638980 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639041 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639086 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639129 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639169 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639215 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1018 05:15:23.639267 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639319 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:23.639350 33299 natgateway.go:117] DescribeNatGateways: { } I1018 05:15:23.639392 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1018 05:15:23.639429 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W1018 05:15:23.639474 33299 api.go:53] PageSize not implemented I1018 05:15:23.639491 33299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1018 05:15:23.639544 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:23.639570 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:23.639588 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:23.639611 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:23.639630 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1018 05:15:23.639692 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.639720 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.639764 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:23.639800 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.639858 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:23.639907 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:23.639938 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1018 05:15:23.639974 33299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1018 05:15:23.640007 33299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok autoscaling-group:bastion.private-shared-ip.example.com ok I1018 05:15:23.640640 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1018 05:15:23.640677 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1018 05:15:23.640719 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I1018 05:15:23.640792 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.640634 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.640841 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok I1018 05:15:23.640784 33299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1018 05:15:23.640880 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.640900 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1018 05:15:23.640934 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I1018 05:15:23.640937 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I1018 05:15:23.641024 33299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok autoscaling-group:nodes.private-shared-ip.example.com ok iam-instance-profile:nodes.private-shared-ip.example.com ok I1018 05:15:23.640962 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W1018 05:15:23.641175 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.641201 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1018 05:15:23.641235 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1018 05:15:23.641292 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.641346 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1018 05:15:23.641424 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.641437 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:23.641467 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1018 05:15:23.641518 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1018 05:15:23.641665 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.641689 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1018 05:15:23.641726 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I1018 05:15:23.641864 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.641918 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1018 05:15:23.641877 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.641933 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1018 05:15:23.641965 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I1018 05:15:23.641996 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.642042 33299 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1018 05:15:23.641968 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1018 05:15:23.642091 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.642109 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:23.642054 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1018 05:15:23.642146 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1018 05:15:23.642176 33299 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I1018 05:15:23.642199 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1018 05:15:23.642216 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I1018 05:15:23.642264 33299 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1018 05:15:23.642280 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1018 05:15:23.642254 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I1018 05:15:23.642312 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok W1018 05:15:23.642334 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:23.642348 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1018 05:15:23.642388 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:23.642476 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:23.642529 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1018 05:15:23.642802 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I1018 05:15:23.643057 33299 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:15:23.643293 33299 internetgateways.go:66] CreateInternetGateway: { } I1018 05:15:23.643339 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:15:23.643383 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:15:23.643434 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:15:23.643476 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:15:23.643506 33299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:15:23.643543 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:15:23.643569 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:15:23.643636 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I1018 05:15:23.668599 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:23.671773 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.671864 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.712260 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.845998 33299 iamrole.go:89] ListRoles: { } I1018 05:15:23.847826 33299 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1018 05:15:23.848157 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.848322 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.848543 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I1018 05:15:23.848544 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.848684 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1018 05:15:23.848722 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:23.848867 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:23.848867 33299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.848958 33299 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:23.849080 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:23.850503 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.849151 33299 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] } I1018 05:15:23.850789 33299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.850935 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.851043 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:23.851212 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.851617 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1018 05:15:23.851730 33299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1018 05:15:23.851831 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:23.851963 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:23.852044 33299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1018 05:15:23.852377 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:23.852488 33299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.853648 33299 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1018 05:15:23.853846 33299 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1018 05:15:23.854641 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:23.853814 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1018 05:15:23.854853 33299 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1018 05:15:23.854954 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1018 05:15:23.855022 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I1018 05:15:23.855164 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:23.855262 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.855943 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.856117 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:23.856219 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1018 05:15:23.856334 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:23.856417 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:23.856345 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1018 05:15:23.856468 33299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:15:23.856339 33299 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I1018 05:15:23.856912 33299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.857003 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:23.857050 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1018 05:15:23.857196 33299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1018 05:15:23.857204 33299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1018 05:15:23.857345 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:23.857406 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:23.857452 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:23.857513 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:23.857568 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:23.857612 33299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1018 05:15:23.857658 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:23.857707 33299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1018 05:15:23.858398 33299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:23.861885 33299 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1018 05:15:23.862128 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.862317 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.862525 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.862750 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.862900 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.863115 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.863265 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.863404 33299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:23.863466 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.863538 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:23.863656 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.863850 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.864187 33299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1018 05:15:23.864310 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1018 05:15:23.864402 33299 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1018 05:15:23.864481 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.864670 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.864817 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.864964 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.865168 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.865384 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.865528 33299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1018 05:15:23.865670 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.865786 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.865874 33299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1018 05:15:23.868037 33299 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1018 05:15:23.868304 33299 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1018 05:15:23.868651 33299 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1018 05:15:23.868707 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:23.868747 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.868803 33299 dns.go:238] Pre-creating DNS records I1018 05:15:23.868836 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:23.874566 33299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1018 05:15:23.877694 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.877787 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:23.916860 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:24.029066 33299 iamrole.go:89] ListRoles: { } I1018 05:15:24.030138 33299 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1018 05:15:24.030273 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.030358 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:15:24.030378 33299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:15:24.030410 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:24.030458 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.030567 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.030660 33299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1018 05:15:24.031936 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.032487 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:24.032634 33299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1018 05:15:24.035456 33299 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1018 05:15:24.035920 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1018 05:15:24.036825 33299 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1018 05:15:24.036917 33299 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1018 05:15:24.036188 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:24.037108 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.037379 33299 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:15:24.037465 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:24.037564 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1018 05:15:24.037787 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:24.037870 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.038008 33299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:15:24.038108 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.042103 33299 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1018 05:15:24.042255 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:24.042427 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:24.042610 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:24.042782 33299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1018 05:15:24.043043 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:24.043320 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:24.043470 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1018 05:15:24.044918 33299 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1018 05:15:24.045174 33299 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1018 05:15:24.045362 33299 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1018 05:15:24.048675 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1018 05:15:24.048740 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1018 05:15:24.048777 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1018 05:15:24.048809 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1018 05:15:24.048841 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1018 05:15:24.048879 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1018 05:15:24.048911 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1018 05:15:24.048941 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1018 05:15:24.048975 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1018 05:15:24.049010 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1018 05:15:24.049047 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1018 05:15:24.049085 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1018 05:15:24.049127 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1018 05:15:24.049159 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1018 05:15:24.049190 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1018 05:15:24.049227 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1018 05:15:24.049259 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:15:24.049296 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1018 05:15:24.049332 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:15:24.049369 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1018 05:15:24.049400 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1018 05:15:24.049431 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1018 05:15:24.049463 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1018 05:15:24.049499 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1018 05:15:24.049534 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:15:24.049569 33299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1018 05:15:24.051076 33299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1018 05:15:24.051108 33299 keypairs.go:95] DescribeKeyPairs: { } I1018 05:15:24.051154 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051211 33299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051269 33299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051326 33299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051374 33299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051433 33299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051475 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051514 33299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051563 33299 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051636 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1018 05:15:24.051683 33299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W1018 05:15:24.051717 33299 api.go:53] PageSize not implemented I1018 05:15:24.051731 33299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1018 05:15:24.051747 33299 loadbalancers.go:34] PageSize not implemented I1018 05:15:24.051759 33299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1018 05:15:24.051773 33299 targetgroups.go:35] PageSize not implemented I1018 05:15:24.051784 33299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1018 05:15:24.051818 33299 zones.go:93] ListHostedZonesPages { } I1018 05:15:24.051835 33299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1018 05:15:24.051867 33299 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1018 05:15:24.051888 33299 iamrole.go:89] ListRoles: { } I1018 05:15:24.051913 33299 internetgateways.go:101] DescribeInternetGateways: { } I1018 05:15:24.051938 33299 routetable.go:53] DescribeRouteTables: { } I1018 05:15:24.051957 33299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I1018 05:15:24.052299 33299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I1018 05:15:24.052367 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1018 05:15:24.052391 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } keypair:key-1 ok I1018 05:15:24.052437 33299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I1018 05:15:24.052494 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } volume:vol-1 ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I1018 05:15:24.052351 33299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.nthsqsresources.longclustername.example.com. ok I1018 05:15:24.052542 33299 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:24.052652 33299 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I1018 05:15:24.052525 33299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1018 05:15:24.052718 33299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I1018 05:15:24.052777 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1018 05:15:24.052840 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1018 05:15:24.053066 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:24.053088 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1018 05:15:24.053133 33299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1018 05:15:24.053183 33299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1018 05:15:24.053290 33299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1018 05:15:24.053305 33299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1018 05:15:24.053375 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1018 05:15:24.053428 33299 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1018 05:15:24.053445 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1018 05:15:24.053431 33299 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1018 05:15:24.053482 33299 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I1018 05:15:24.053540 33299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:24.053602 33299 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1018 05:15:24.053617 33299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1018 05:15:24.053654 33299 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I1018 05:15:24.053724 33299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1018 05:15:24.053801 33299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1018 05:15:24.053871 33299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1018 05:15:24.054156 33299 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.41s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 55.941s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.023s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I1018 05:14:06.794230 24428 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.071s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.025s === RUN TestPodController I1018 05:14:11.868925 28686 pod.go:59] starting pod controller I1018 05:14:11.869209 28686 stoppable.go:62] shutting down controller I1018 05:14:11.869323 28686 pod.go:65] shutting down pod controller --- PASS: TestPodController (0.00s) PASS I1018 05:14:11.869240 28686 pod.go:107] Got stop signal ok k8s.io/kops/dns-controller/pkg/watchers 0.051s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.020s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5e00} route53_test.go:93: Got zone list: [0xc0000a5e30] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc000012090} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5f00} route53_test.go:93: Got zone list: [0xc0000a5f30] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000120a0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000a5fd0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0002bb500 0xc0000120b8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0130} route53_test.go:93: Got zone list: [0xc0002e0160] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000120d0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0260} route53_test.go:93: Got zone list: [0xc0002e0290] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000120f0} route53_test.go:187: Successfully added resource record set: {0xc000112600 0xc000012110} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0510} route53_test.go:93: Got zone list: [0xc0002e0540] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc000012198} route53_test.go:199: Successfully added resource record set: {0xc000112780 0xc0000121b8} route53_test.go:118: Got 1 recordsets: [0xc0002e06e0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0810} route53_test.go:93: Got zone list: [0xc0002e0840] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc000012250} route53_test.go:221: Successfully added resource record set: {0xc000112900 0xc000012270} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000112900 0xc000012270}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0ba0} route53_test.go:93: Got zone list: [0xc0002e0bd0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc000012328} route53_test.go:246: Successfully removed resource set {0xc000112b00 0xc000012348} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e0e50} route53_test.go:93: Got zone list: [0xc0002e0e80] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000123d0} route53_test.go:264: Successfully removed resource set {0xc000112c80 0xc0000123f0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e1130} route53_test.go:93: Got zone list: [0xc0002e1160] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc000012480} commontests.go:192: Successfully added resource record set: {0xc000112e00 0xc0000124a0} commontests.go:45: Correctly replaced resource record {0xc000112e00 0xc0000124a0} -> {0xc000112f00 0xc0000124f0} commontests.go:142: Got 1 recordsets: [0xc0002e1470] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e1640} route53_test.go:93: Got zone list: [0xc0002e1670] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000125a0} commontests.go:192: Successfully added resource record set: {0xc000113180 0xc0000125c0} commontests.go:70: Correctly replaced resource record {0xc000113180 0xc0000125c0} -> {0xc000113280 0xc000012610} commontests.go:142: Got 1 recordsets: [0xc0002e1980] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e1ba0} route53_test.go:93: Got zone list: [0xc0002e1bd0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000126d0} commontests.go:192: Successfully added resource record set: {0xc000113500 0xc0000126f0} commontests.go:142: Got 2 recordsets: [0xc0002e1e40 0xc0002e1e60] commontests.go:142: Got 2 recordsets: [0xc0002e1ed0 0xc0002e1ef0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002f0090} route53_test.go:93: Got zone list: [0xc0002f00c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002bb320 0xc0000127f0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.025s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.034s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc120} clouddns_test.go:71: Got zone list: [0xc0000103a8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000010378} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc220} clouddns_test.go:71: Got zone list: [0xc000010450] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000010420} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc320} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000305180 0xc0000104f8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc3e0} clouddns_test.go:71: Got zone list: [0xc0000105a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000010570} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc500} clouddns_test.go:71: Got zone list: [0xc000010828] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc0000107f8} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc640} clouddns_test.go:71: Got zone list: [0xc000010978] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000010948} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc7a0} clouddns_test.go:71: Got zone list: [0xc000010f60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000010f18} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cc900} clouddns_test.go:71: Got zone list: [0xc0000114a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000011470} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cca40} clouddns_test.go:71: Got zone list: [0xc0000115f0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc0000115c0} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004ccba0} clouddns_test.go:71: Got zone list: [0xc0000117a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000011770} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004ccd00} clouddns_test.go:71: Got zone list: [0xc000011cc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc000011c68} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cce80} clouddns_test.go:71: Got zone list: [0xc00041e348] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc00041e2e8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000012038 0xc0004cd060} clouddns_test.go:71: Got zone list: [0xc00041eac8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000304fc0 0xc00041ea68} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.147s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.126s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.143s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I1018 05:14:28.741307 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.741554 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.741636 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.741692 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.741749 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.741787 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.741809 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.741843 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.741877 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.751776 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I1018 05:14:28.755631 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.755830 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.755869 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.755903 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.755935 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.755968 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.755986 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.756010 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.756028 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.757252 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I1018 05:14:28.760932 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.761119 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.761153 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.761180 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.761209 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.761235 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.761264 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.761291 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.761309 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.762505 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.764133 33367 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I1018 05:14:28.764155 33367 assetstore.go:111] bin/containerd I1018 05:14:28.764164 33367 assetstore.go:111] bin/containerd-shim I1018 05:14:28.764171 33367 assetstore.go:111] bin/containerd-shim-runc-v1 I1018 05:14:28.764178 33367 assetstore.go:111] bin/containerd-shim-runc-v2 I1018 05:14:28.764185 33367 assetstore.go:111] bin/containerd-stress I1018 05:14:28.764192 33367 assetstore.go:111] bin/ctr I1018 05:14:28.764243 33367 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I1018 05:14:28.764261 33367 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I1018 05:14:28.767961 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.768148 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.768184 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.768212 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.768241 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.768277 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.768297 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.768318 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.768337 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.769436 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.770871 33367 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I1018 05:14:28.771796 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.771960 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.771987 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.772011 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.772037 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.772112 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.772131 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.772152 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.772173 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.773210 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.774739 33367 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I1018 05:14:28.775516 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.775702 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.775735 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.775762 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.775790 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.775844 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.775863 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.775886 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.775903 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.777053 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.778610 33367 assetstore.go:108] Matching assets for "^docker/": I1018 05:14:28.778629 33367 assetstore.go:111] docker/containerd I1018 05:14:28.778637 33367 assetstore.go:111] docker/containerd-shim I1018 05:14:28.778644 33367 assetstore.go:111] docker/ctr I1018 05:14:28.778652 33367 assetstore.go:111] docker/docker I1018 05:14:28.778659 33367 assetstore.go:111] docker/docker-init I1018 05:14:28.778665 33367 assetstore.go:111] docker/docker-proxy I1018 05:14:28.778672 33367 assetstore.go:111] docker/dockerd I1018 05:14:28.778678 33367 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I1018 05:14:28.783474 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.783606 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.783635 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.783662 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.783688 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.783730 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.783744 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.783761 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.783775 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.784654 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.785895 33367 assetstore.go:108] Matching assets for "^docker/": I1018 05:14:28.785909 33367 assetstore.go:111] docker/containerd I1018 05:14:28.785916 33367 assetstore.go:111] docker/containerd-shim I1018 05:14:28.785923 33367 assetstore.go:111] docker/ctr I1018 05:14:28.785930 33367 assetstore.go:111] docker/docker I1018 05:14:28.785938 33367 assetstore.go:111] docker/docker-init I1018 05:14:28.785948 33367 assetstore.go:111] docker/docker-proxy I1018 05:14:28.785959 33367 assetstore.go:111] docker/dockerd I1018 05:14:28.785968 33367 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I1018 05:14:28.788608 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.788736 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.788768 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.788790 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.788818 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.788856 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.788872 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.788890 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.788903 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.789731 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.790911 33367 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I1018 05:14:28.791027 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.791124 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.791143 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.791162 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.791187 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.791217 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.791231 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.791248 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.791260 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.792029 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:28.793074 33367 assetstore.go:108] Matching assets for "^docker/": I1018 05:14:28.793087 33367 assetstore.go:111] docker/docker I1018 05:14:28.793094 33367 assetstore.go:111] docker/docker-containerd I1018 05:14:28.793100 33367 assetstore.go:111] docker/docker-containerd-ctr I1018 05:14:28.793105 33367 assetstore.go:111] docker/docker-containerd-shim I1018 05:14:28.793111 33367 assetstore.go:111] docker/docker-init I1018 05:14:28.793116 33367 assetstore.go:111] docker/docker-proxy I1018 05:14:28.793121 33367 assetstore.go:111] docker/docker-runc I1018 05:14:28.793129 33367 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I1018 05:14:28.797179 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.797336 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.797370 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.797394 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.797426 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.797451 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.797466 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.797487 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.797503 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.799612 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I1018 05:14:28.801456 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.801592 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.801627 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.801666 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.801688 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.801706 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.801720 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.801741 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.801754 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.803086 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I1018 05:14:28.804767 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.804870 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.804892 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.804913 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.804955 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.804976 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.804989 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.805017 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.805031 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.806271 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I1018 05:14:28.811643 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.812207 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.812238 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.812274 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.812307 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.812342 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.812354 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.812372 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.812397 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.813800 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I1018 05:14:28.820682 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.820813 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.820837 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.820858 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.820881 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.820899 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.820912 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.820927 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.820939 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.822291 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I1018 05:14:28.827149 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.827286 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.827311 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.827333 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.827356 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.827375 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.827387 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.827405 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.827418 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.828604 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I1018 05:14:28.833652 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.833775 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.833810 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.833855 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.833898 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.833932 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.833966 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.834007 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.834031 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.837734 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I1018 05:14:28.850929 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:28.851188 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.851241 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:28.851294 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:28.851347 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:28.851387 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:28.851417 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:28.851454 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:28.851483 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:28.854511 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.64s) === RUN TestKubeAPIServerBuilderARM64 I1018 05:14:29.490701 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.490881 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.490923 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.490958 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.490993 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.491022 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.491044 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.491068 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.491087 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.493089 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I1018 05:14:29.567202 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.567374 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.567418 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.567459 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.567507 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.567542 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.567575 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.567876 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.567963 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.571027 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I1018 05:14:29.577017 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.577160 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.577190 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.577217 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.577244 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.577266 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.577283 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.577304 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.577319 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.578960 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I1018 05:14:29.647497 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.647658 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.647692 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.647722 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.647751 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.647782 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.647800 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.647824 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.647842 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.649652 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I1018 05:14:29.718761 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.718927 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.718963 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.718993 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.719026 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.719051 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.719070 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.719094 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.719111 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.720720 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I1018 05:14:29.726227 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.726376 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.726407 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.726436 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.726465 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.726489 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.726505 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.726525 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.726540 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.728289 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I1018 05:14:29.794802 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.794979 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.795017 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.795051 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.795081 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.795110 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.795129 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.795152 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.795168 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.796823 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I1018 05:14:29.863429 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.863626 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.863667 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.863702 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.863739 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.863769 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.863788 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.863814 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.863833 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.865884 33367 zones.go:93] ListHostedZonesPages { } W1018 05:14:29.868831 33367 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I1018 05:14:29.870960 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.871124 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.871159 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.871192 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.871225 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.871252 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.871273 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.871298 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.871319 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.873364 33367 zones.go:93] ListHostedZonesPages { } W1018 05:14:29.938053 33367 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I1018 05:14:29.939819 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:29.939976 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:29.940012 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:29.940040 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:29.940071 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:29.940098 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:29.940115 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:29.940137 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:29.940155 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:29.941812 33367 zones.go:93] ListHostedZonesPages { } W1018 05:14:30.007237 33367 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I1018 05:14:30.009041 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.009205 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.009240 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.009272 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.009305 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.009330 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.009347 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.009368 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.009385 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.010966 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:30.012641 33367 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.00s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I1018 05:14:30.014423 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.014531 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.014570 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.014594 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.014618 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.014641 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.014658 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.014678 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.014692 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.015952 33367 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I1018 05:14:30.019138 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.019245 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.019282 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.019305 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.019332 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.019354 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.019371 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.019391 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.019406 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.020598 33367 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I1018 05:14:30.023687 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.023809 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.023835 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.023858 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.023882 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.023904 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.023919 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.023940 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.023953 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.025447 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:30.026973 33367 assetstore.go:108] Matching assets for "protokube$": I1018 05:14:30.026992 33367 assetstore.go:111] /opt/kops/bin/protokube I1018 05:14:30.027013 33367 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I1018 05:14:30.027034 33367 assetstore.go:108] Matching assets for "channels$": I1018 05:14:30.027045 33367 assetstore.go:111] /opt/kops/bin/channels I1018 05:14:30.027052 33367 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I1018 05:14:30.028422 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.028541 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.028565 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.028587 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.028613 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.028634 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.028652 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.028672 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.028685 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.030153 33367 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I1018 05:14:30.032184 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.032297 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.032325 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.032347 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.032371 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.032392 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.032409 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.032430 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.032444 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.033912 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:30.035401 33367 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I1018 05:14:30.035766 33367 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:30.035880 33367 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:30.035905 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:30.035927 33367 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:30.035951 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:30.035971 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:30.035991 33367 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:30.036011 33367 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:30.036026 33367 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:30.038644 33367 zones.go:93] ListHostedZonesPages { } I1018 05:14:30.041885 33367 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 1.362s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.074s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.082s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.078s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.091s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E1018 05:14:11.609300 28363 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E1018 05:14:11.609406 28363 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.046s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I1018 05:14:22.706894 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707035 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707106 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707165 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707230 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I1018 05:14:22.707564 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707613 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707678 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707735 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707803 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707855 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707914 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.707964 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708024 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708078 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708126 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708179 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708243 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708305 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708355 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708423 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708475 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708535 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1018 05:14:22.708585 31795 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I1018 05:14:22.708839 31795 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I1018 05:14:22.708921 31795 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.218s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I1018 05:14:11.867027 28683 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1018 05:14:11.868124 28683 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1018 05:14:11.869815 28683 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1018 05:14:11.870796 28683 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1018 05:14:11.872024 28683 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1018 05:14:11.872414 28683 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1018 05:14:11.872858 28683 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1018 05:14:11.873023 28683 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1018 05:14:11.873586 28683 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I1018 05:14:11.873662 28683 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I1018 05:14:11.873970 28683 health.go:33] status conditions not found for /v1, Kind=Namespace I1018 05:14:11.874178 28683 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I1018 05:14:11.874266 28683 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I1018 05:14:11.874502 28683 health.go:33] status conditions not found for /v1, Kind=ConfigMap I1018 05:14:11.874708 28683 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I1018 05:14:11.874773 28683 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I1018 05:14:11.875511 28683 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I1018 05:14:11.875562 28683 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.03s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.069s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W1018 05:14:12.099927 28379 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.41s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W1018 05:14:12.101043 28379 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W1018 05:14:12.102213 28379 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W1018 05:14:12.103255 28379 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W1018 05:14:12.104341 28379 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W1018 05:14:12.105717 28379 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W1018 05:14:12.106703 28379 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E1018 05:14:12.107389 28379 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E1018 05:14:12.107402 28379 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E1018 05:14:12.107410 28379 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E1018 05:14:12.107419 28379 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.523s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.004s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.102s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.034s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.263s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.083s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.005s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.013s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.088s === RUN TestFlagToFalse I1018 05:14:12.116772 28921 featureflag_test.go:32] Created flag Unittest1 I1018 05:14:12.116842 28921 featureflag.go:160] FeatureFlag "UnitTest1"=false I1018 05:14:12.116857 28921 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I1018 05:14:12.116938 28921 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.019s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.132s === RUN TestRenderInstanceGroupZones W1018 05:14:14.816853 29766 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.095s === RUN TestWarmPoolOnlyRoll I1018 05:14:38.772644 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.772715 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.772733 33653 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I1018 05:14:38.773093 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.773113 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.774292 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.774335 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.779622 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.779644 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.779659 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.779668 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:38.781087 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:38.781634 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.781657 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.782809 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.782842 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.788119 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.788147 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:38.788618 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:38.788639 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.789814 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.789836 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.795060 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.795089 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.795100 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.795110 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:38.795703 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:38.796138 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.796161 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.797349 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.797376 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.802629 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.802657 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:38.803136 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:38.803157 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.804352 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.804382 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.809610 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.809638 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:38.810128 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:38.810149 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.811325 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.811371 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.816604 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.816655 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.816690 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.816709 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:38.817718 33653 instancegroups.go:402] Draining the node: "node-2a.local". I1018 05:14:38.818576 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:38.818617 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.819826 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.819859 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.825102 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.825152 33653 instancegroups.go:402] Draining the node: "node-2b.local". I1018 05:14:38.826035 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:38.826074 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.827284 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.827331 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.832555 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.832607 33653 instancegroups.go:402] Draining the node: "node-2c.local". I1018 05:14:38.833516 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:38.833555 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.834757 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.834793 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.840060 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.840099 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I1018 05:14:38.840663 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.840706 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.841891 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.841955 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.841994 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.842008 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.842034 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.843197 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.843248 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.843263 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:38.843290 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.844473 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.844509 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.844569 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.844594 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.844625 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.845827 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.845879 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.845894 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:38.845921 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.847114 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.847165 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.847180 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:38.847207 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.848393 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.848421 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.848470 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.848484 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:38.848511 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.849709 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.849760 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.849776 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:38.849803 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.850999 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. W1018 05:14:38.851050 33653 instancegroups.go:399] Not draining cluster nodes as 'cloudonly' flag is set. I1018 05:14:38.851065 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:38.851093 33653 instancegroups.go:438] waiting for 1ms after terminating instance W1018 05:14:38.852289 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1018 05:14:38.852321 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I1018 05:14:38.852608 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.852643 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.853851 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.853894 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.853917 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.853967 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.853993 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.854011 33653 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.854030 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:38.854773 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:38.855621 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.855660 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.856859 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.856900 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.856922 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.856963 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:38.857781 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:38.857819 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.859017 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.859059 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.859082 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.859104 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.859126 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.859140 33653 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.859153 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:38.860277 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:38.861069 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.861108 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.862315 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.862357 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.862380 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.862420 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:38.863257 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:38.863294 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.864495 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.864536 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.864558 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.864598 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:38.865506 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:38.865545 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.866744 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.866785 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.866806 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.866822 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.866840 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.866860 33653 instancegroups.go:481] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.866875 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:38.867819 33653 instancegroups.go:402] Draining the node: "node-2a.local". I1018 05:14:38.868678 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:38.868721 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.869923 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.869964 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.869991 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.870031 33653 instancegroups.go:402] Draining the node: "node-2b.local". I1018 05:14:38.870851 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:38.870889 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.872083 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.872124 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.872146 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.872186 33653 instancegroups.go:402] Draining the node: "node-2c.local". I1018 05:14:38.873052 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:38.873091 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.874293 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.874335 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. W1018 05:14:38.874357 33653 instancegroups.go:481] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1018 05:14:38.874375 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I1018 05:14:38.874646 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I1018 05:14:38.874926 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.874981 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.876146 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.876181 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.881422 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.881509 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.881549 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.881575 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:38.882523 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:38.883498 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.883546 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.884725 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.884755 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.890002 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.890059 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:38.891031 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:38.891072 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.892279 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.892314 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.897553 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.897594 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.897614 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.897628 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:38.898864 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:38.899743 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.899780 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.900988 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.901023 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.906264 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.906316 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:38.907333 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:38.907373 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.908556 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.908591 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.913851 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.913903 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:38.914902 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:38.914956 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.916126 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.916161 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.921439 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.921471 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.921515 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.921531 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:38.922721 33653 instancegroups.go:402] Draining the node: "node-2a.local". I1018 05:14:38.923615 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:38.923653 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.924859 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.924917 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.930254 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.930305 33653 instancegroups.go:402] Draining the node: "node-2b.local". I1018 05:14:38.931319 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:38.931362 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.932575 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.932615 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.937880 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.937940 33653 instancegroups.go:402] Draining the node: "node-2c.local". I1018 05:14:38.938967 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:38.939007 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.940219 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.940254 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.945559 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.945594 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I1018 05:14:38.945752 33653 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I1018 05:14:38.946299 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.946332 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.947515 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.947555 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.947577 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I1018 05:14:38.948048 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.948083 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.949300 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.949338 33653 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1018 05:14:38.949361 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I1018 05:14:38.949642 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.949675 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.949691 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I1018 05:14:38.949878 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.949906 33653 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1018 05:14:38.949922 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I1018 05:14:38.950219 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.950252 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.951394 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.951443 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.956739 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.956801 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.956825 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.956846 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:38.957730 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:38.958668 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.958709 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.959924 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.959970 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.959991 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I1018 05:14:38.960440 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:38.960473 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.961688 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.961731 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.967058 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.967124 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.967154 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.967176 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:38.968057 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:38.969017 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:38.969056 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.970277 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.970321 33653 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1018 05:14:38.970366 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I1018 05:14:38.970709 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.970736 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.970750 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:38.971970 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:38.973848 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.973887 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.975080 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.975117 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.980442 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.980499 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:38.981812 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:38.981859 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.983063 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.983099 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.988358 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.988410 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:38.989241 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:38.989279 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.990509 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.990544 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:38.995829 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.995863 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.03s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I1018 05:14:38.996153 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.996184 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.996202 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I1018 05:14:38.996417 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.996445 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.996463 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I1018 05:14:38.996743 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.996774 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:38.996789 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I1018 05:14:38.996963 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:38.996991 33653 instancegroups.go:505] Cluster validated. I1018 05:14:38.997004 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:38.997988 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:38.998741 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:38.998781 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:38.999981 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.000028 33653 instancegroups.go:525] Cluster did not pass validation within deadline: testing failure. E1018 05:14:39.000049 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I1018 05:14:39.000278 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.000312 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.000326 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:39.001372 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.003342 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.003495 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.004986 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.005052 33653 instancegroups.go:515] Cluster did not validate within deadline: testing validation error. E1018 05:14:39.005094 33653 instancegroups.go:477] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.01s) === RUN TestRollingUpdateFlappingValidation I1018 05:14:39.005529 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:39.005558 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.006775 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.006834 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.007976 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.013213 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.014499 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.019779 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.020949 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.026205 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.026270 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.026303 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.026318 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:39.027174 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:39.027986 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:39.028022 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.029249 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.029294 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.034582 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.034637 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:39.035491 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:39.035528 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.036741 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.036797 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.042068 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.042107 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.042136 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.042151 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:39.043129 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.043916 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.043953 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.045150 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.045201 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.050520 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.050574 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.051402 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.051437 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.052670 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.052721 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.058029 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.058082 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:39.058957 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:39.058993 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.060219 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.060273 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.065559 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.065591 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.065621 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.065635 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:39.066613 33653 instancegroups.go:402] Draining the node: "node-2a.local". I1018 05:14:39.067391 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:39.067427 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.068626 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.068677 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.073984 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.074045 33653 instancegroups.go:402] Draining the node: "node-2b.local". I1018 05:14:39.074904 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:39.074963 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.076165 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.076215 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.081530 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.081583 33653 instancegroups.go:402] Draining the node: "node-2c.local". I1018 05:14:39.082419 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:39.082455 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.083689 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.083740 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.089033 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.089066 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I1018 05:14:39.089368 33653 instancegroups.go:595] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1018 05:14:39.089401 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.090633 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.090678 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.091856 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.093028 33653 instancegroups.go:528] Cluster did not pass validation, will retry in "1ms": testing failure. I1018 05:14:39.094193 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.099461 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.099526 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.099558 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.099572 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:39.100326 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:39.101136 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:39.101175 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.102379 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.102418 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.107711 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.107764 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:39.108630 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:39.108667 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.109874 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.109911 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.115165 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.115204 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.115228 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.115243 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:39.116228 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.116983 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.117019 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.118239 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.118296 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.123586 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.123639 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.124464 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.124501 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.125730 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.125767 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.131041 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.131094 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:39.131952 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:39.131988 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.133210 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.133247 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.138508 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.138539 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.138562 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.138577 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:39.139568 33653 instancegroups.go:402] Draining the node: "node-2a.local". I1018 05:14:39.140329 33653 instancegroups.go:593] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1018 05:14:39.140364 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.141593 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.141659 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.146928 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.146983 33653 instancegroups.go:402] Draining the node: "node-2b.local". I1018 05:14:39.147833 33653 instancegroups.go:593] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1018 05:14:39.147874 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.149081 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.149117 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.154366 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.154419 33653 instancegroups.go:402] Draining the node: "node-2c.local". I1018 05:14:39.155308 33653 instancegroups.go:593] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1018 05:14:39.155346 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.156591 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.156628 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.161890 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.161924 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I1018 05:14:39.162515 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.162543 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.162557 33653 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1018 05:14:39.163301 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.164075 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.164108 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.165308 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.165345 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.170622 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.170673 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.171523 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.171558 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.172762 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.172799 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.178083 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.178115 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I1018 05:14:39.178395 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.178424 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.178437 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:39.179184 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:39.179966 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:39.180001 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.181234 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.181271 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.186541 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.186590 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:39.187455 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:39.187491 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.188692 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.188729 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.194011 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.194043 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I1018 05:14:39.194372 33653 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled W1018 05:14:39.194422 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1018 05:14:39.194444 33653 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled W1018 05:14:39.194462 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1018 05:14:39.194487 33653 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled W1018 05:14:39.194497 33653 instancegroups.go:470] Not validating cluster as cloudonly flag is set. I1018 05:14:39.194517 33653 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1018 05:14:39.194532 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I1018 05:14:39.194809 33653 instancegroups.go:572] Detaching instance "bastion-1a", in group "bastion-1". I1018 05:14:39.194866 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:39.196104 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.196142 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.201409 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.201447 33653 instancegroups.go:187] Rolling updates for InstanceGroup bastion-1 are disabled I1018 05:14:39.201499 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.201521 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.201537 33653 instancegroups.go:313] Tainting 2 nodes in "master-1" instancegroup. I1018 05:14:39.202263 33653 instancegroups.go:187] Rolling updates for InstanceGroup master-1 are disabled I1018 05:14:39.202292 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.202311 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.202324 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:39.203208 33653 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1018 05:14:39.203268 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:39.204481 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.204515 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.209779 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.209819 33653 instancegroups.go:187] Rolling updates for InstanceGroup node-1 are disabled I1018 05:14:39.209834 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.209852 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.209866 33653 instancegroups.go:313] Tainting 3 nodes in "node-2" instancegroup. I1018 05:14:39.210808 33653 instancegroups.go:570] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I1018 05:14:39.210867 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:39.212051 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.212086 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:39.217361 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.217399 33653 instancegroups.go:187] Rolling updates for InstanceGroup node-2 are disabled I1018 05:14:39.217416 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I1018 05:14:39.217644 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.217677 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.217691 33653 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1018 05:14:39.219888 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.220695 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.220741 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.221930 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.221974 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.222017 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:39.222114 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.223386 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:39.223684 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.223727 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.224948 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.225019 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.325561 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.325619 33653 instancegroups.go:402] Draining the node: "node-1e.local". I1018 05:14:39.325688 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:39.326960 33653 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1018 05:14:39.327286 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:39.327336 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.328526 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.328570 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.328611 33653 instancegroups.go:402] Draining the node: "node-1f.local". I1018 05:14:39.329437 33653 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1018 05:14:39.329480 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.330694 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.330756 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.431215 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.431265 33653 instancegroups.go:402] Draining the node: "node-1g.local". I1018 05:14:39.432123 33653 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1018 05:14:39.432168 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.433378 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.433421 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.433439 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I1018 05:14:39.433655 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.433687 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.433702 33653 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1018 05:14:39.435647 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.435712 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.436921 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.437224 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.437267 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.438458 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.438520 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.539073 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.539153 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:39.539238 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:39.540601 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:39.540888 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:39.540934 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.542128 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.542174 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.542216 33653 instancegroups.go:402] Draining the node: "node-1e.local". I1018 05:14:39.543040 33653 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1018 05:14:39.543083 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.544282 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.544342 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.644846 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.644910 33653 instancegroups.go:402] Draining the node: "node-1f.local". I1018 05:14:39.645858 33653 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1018 05:14:39.645907 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.647162 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.647230 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.647258 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I1018 05:14:39.647627 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.647687 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.647708 33653 instancegroups.go:313] Tainting 7 nodes in "master-1" instancegroup. I1018 05:14:39.650215 33653 instancegroups.go:402] Draining the node: "master-1a.local". I1018 05:14:39.650991 33653 instancegroups.go:593] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1018 05:14:39.651038 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.652202 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.652252 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.652297 33653 instancegroups.go:402] Draining the node: "master-1c.local". I1018 05:14:39.652385 33653 instancegroups.go:402] Draining the node: "master-1b.local". I1018 05:14:39.653731 33653 instancegroups.go:593] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I1018 05:14:39.654088 33653 instancegroups.go:593] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1018 05:14:39.654140 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.655304 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.655383 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.755848 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.755933 33653 instancegroups.go:402] Draining the node: "master-1e.local". I1018 05:14:39.756006 33653 instancegroups.go:402] Draining the node: "master-1d.local". I1018 05:14:39.759736 33653 instancegroups.go:593] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I1018 05:14:39.759995 33653 instancegroups.go:593] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I1018 05:14:39.760034 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.761260 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.761313 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.761362 33653 instancegroups.go:402] Draining the node: "master-1f.local". I1018 05:14:39.762370 33653 instancegroups.go:593] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I1018 05:14:39.762425 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.763615 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.763715 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.863977 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.864057 33653 instancegroups.go:402] Draining the node: "master-1g.local". I1018 05:14:39.865296 33653 instancegroups.go:593] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I1018 05:14:39.865351 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.866572 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.866624 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.866645 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I1018 05:14:39.866944 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.866981 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.866996 33653 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1018 05:14:39.869006 33653 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1018 05:14:39.869077 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:39.870290 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.870341 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.870362 33653 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1018 05:14:39.870427 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:39.871639 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.871684 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.871734 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:39.871783 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:39.873238 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:39.873649 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:39.873702 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.874918 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.875007 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.975668 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.975825 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:39.975902 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:39.977398 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:39.977745 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:39.977792 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.979010 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.979059 33653 instancegroups.go:505] Cluster validated. I1018 05:14:39.979099 33653 instancegroups.go:402] Draining the node: "node-1e.local". I1018 05:14:39.980020 33653 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1018 05:14:39.980070 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:39.981303 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:39.981363 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.081904 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.081959 33653 instancegroups.go:402] Draining the node: "node-1f.local". I1018 05:14:40.082946 33653 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1018 05:14:40.082994 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.084201 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.084243 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.084263 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I1018 05:14:40.084561 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.084597 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.084612 33653 instancegroups.go:313] Tainting 6 nodes in "node-1" instancegroup. I1018 05:14:40.086849 33653 instancegroups.go:570] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1018 05:14:40.086897 33653 instancegroups.go:570] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1018 05:14:40.086943 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:40.088154 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.088198 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.088246 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.088350 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.089719 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.090006 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.090053 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.091286 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.091356 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.191912 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.191965 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:40.191989 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:40.193402 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:40.193730 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:40.193773 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.195002 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.195046 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.195085 33653 instancegroups.go:402] Draining the node: "node-1e.local". I1018 05:14:40.195911 33653 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1018 05:14:40.195959 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.197172 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.197233 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.298147 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.298201 33653 instancegroups.go:402] Draining the node: "node-1f.local". I1018 05:14:40.299060 33653 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1018 05:14:40.299113 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.300341 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.300384 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.300402 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I1018 05:14:40.300610 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.300637 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.300649 33653 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1018 05:14:40.301405 33653 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1018 05:14:40.301438 33653 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I1018 05:14:40.301475 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:40.302705 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.302745 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.308044 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.308108 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.308196 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.309582 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.309625 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.309896 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.309932 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.310870 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.310904 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.316219 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.316258 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I1018 05:14:40.316544 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.316574 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.316588 33653 instancegroups.go:313] Tainting 2 nodes in "node-1" instancegroup. I1018 05:14:40.317430 33653 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E1018 05:14:40.317495 33653 instancegroups.go:162] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I1018 05:14:40.317508 33653 instancegroups.go:570] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E1018 05:14:40.317546 33653 instancegroups.go:162] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I1018 05:14:40.317580 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:40.318789 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.318845 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.324150 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.324207 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.324243 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.325612 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.325653 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.325918 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.325952 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.326881 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.326917 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.332534 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.332568 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I1018 05:14:40.332802 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.332836 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.332850 33653 instancegroups.go:313] Tainting 4 nodes in "node-1" instancegroup. I1018 05:14:40.334125 33653 instancegroups.go:570] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1018 05:14:40.334185 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:40.335377 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.335426 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.335441 33653 instancegroups.go:570] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1018 05:14:40.335499 33653 instancegroups.go:174] waiting for 1ms after detaching instance I1018 05:14:40.336718 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.336762 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.336810 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:40.336876 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.336949 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.338540 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:40.338591 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.338937 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.338990 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.339266 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.339317 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.339817 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.440806 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.440861 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:40.441764 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:40.441814 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.443049 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.443093 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.443110 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I1018 05:14:40.443327 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.443358 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.443372 33653 instancegroups.go:313] Tainting 7 nodes in "node-1" instancegroup. I1018 05:14:40.445514 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.446265 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.446308 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.447525 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.447572 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.447616 33653 instancegroups.go:402] Draining the node: "node-1e.local". I1018 05:14:40.447691 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:40.449015 33653 instancegroups.go:593] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1018 05:14:40.449374 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:40.449421 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.450656 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.450720 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.551645 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.551716 33653 instancegroups.go:402] Draining the node: "node-1g.local". I1018 05:14:40.551801 33653 instancegroups.go:402] Draining the node: "node-1f.local". I1018 05:14:40.553207 33653 instancegroups.go:593] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1018 05:14:40.553498 33653 instancegroups.go:593] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1018 05:14:40.553542 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.554759 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.554808 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.554852 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.555725 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.555775 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.557016 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.557078 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.658013 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.658086 33653 instancegroups.go:402] Draining the node: "node-1d.local". I1018 05:14:40.659035 33653 instancegroups.go:593] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1018 05:14:40.659084 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.660309 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.660368 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.660389 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I1018 05:14:40.660680 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.660709 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.660724 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1018 05:14:40.660746 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1018 05:14:40.660761 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1018 05:14:40.660780 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I1018 05:14:40.660932 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.660958 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.660971 33653 instancegroups.go:313] Tainting 3 nodes in "node-1" instancegroup. I1018 05:14:40.662039 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1018 05:14:40.662073 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1018 05:14:40.662090 33653 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1018 05:14:40.662134 33653 instancegroups.go:402] Draining the node: "node-1a.local". I1018 05:14:40.662925 33653 instancegroups.go:593] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1018 05:14:40.662963 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.664198 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.664239 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.669485 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.669539 33653 instancegroups.go:402] Draining the node: "node-1b.local". I1018 05:14:40.670388 33653 instancegroups.go:593] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1018 05:14:40.670427 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.671664 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.671709 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.676965 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.677020 33653 instancegroups.go:402] Draining the node: "node-1c.local". I1018 05:14:40.677853 33653 instancegroups.go:593] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1018 05:14:40.677891 33653 instancegroups.go:438] waiting for 1ms after terminating instance I1018 05:14:40.679127 33653 instancegroups.go:472] Validating the cluster. I1018 05:14:40.679161 33653 instancegroups.go:508] Cluster validated; revalidating in 5ms to make sure it does not flap. I1018 05:14:40.684417 33653 instancegroups.go:505] Cluster validated. I1018 05:14:40.684453 33653 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.995s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.033s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.060s === RUN TestParse E1018 05:14:13.530834 29377 versions.go:96] unable to parse Kubernetes version "" E1018 05:14:13.530942 29377 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.047s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.111s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.18s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.05s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.337s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.075s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.087s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.072s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I1018 05:14:21.860768 31608 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.121s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I1018 05:14:25.710099 32821 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W1018 05:14:25.752321 32821 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W1018 05:14:25.757049 32821 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.159s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I1018 05:14:25.121959 32654 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.03s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.03s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.157s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.04s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.04s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.185s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.060s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.202s === RUN TestServerGroupBuilder I1018 05:14:26.663925 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.677472 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.690333 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.709163 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.801517 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.818177 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.844286 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.863090 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.872468 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.940376 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.947234 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.953505 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.959665 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.971556 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.979334 33065 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.32s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.397s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.066s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.025s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.066s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.77s) --- PASS: TestIssueCert/ca (0.74s) --- PASS: TestIssueCert/client (0.39s) --- PASS: TestIssueCert/clientOneYear (0.63s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.785s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.048s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I1018 05:14:26.977375 33183 routetable.go:53] DescribeRouteTables: { } W1018 05:14:26.977526 33183 tags.go:67] cluster tag not found on route-table:rtb-1234 I1018 05:14:26.977544 33183 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I1018 05:14:26.977700 33183 iaminstanceprofile.go:193] ListInstanceProfiles: { } W1018 05:14:26.977780 33183 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I1018 05:14:26.977871 33183 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I1018 05:14:26.977975 33183 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I1018 05:14:26.978075 33183 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I1018 05:14:26.978178 33183 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I1018 05:14:26.978312 33183 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I1018 05:14:26.978355 33183 volumes.go:128] DescribeVolumes: { } W1018 05:14:26.978390 33183 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I1018 05:14:26.978562 33183 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:26.978794 33183 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:26.978846 33183 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:26.979370 33183 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:26.979470 33183 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:26.979542 33183 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:26.979574 33183 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:26.979635 33183 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:26.979670 33183 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:26.979712 33183 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I1018 05:14:26.979754 33183 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I1018 05:14:26.979795 33183 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I1018 05:14:26.979928 33183 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I1018 05:14:26.979990 33183 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I1018 05:14:26.980103 33183 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I1018 05:14:26.980172 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I1018 05:14:26.980221 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I1018 05:14:26.980271 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I1018 05:14:26.980314 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I1018 05:14:26.980472 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I1018 05:14:26.981076 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I1018 05:14:26.981125 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I1018 05:14:26.981157 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I1018 05:14:26.981194 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I1018 05:14:26.981226 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I1018 05:14:26.981265 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I1018 05:14:26.981301 33183 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.078s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.048s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.173s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.067s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.015s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.052s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.027s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.026s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.02s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.01s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.124s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.071s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.028s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W1018 05:14:17.959244 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959326 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959337 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959348 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959357 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959367 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959381 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959391 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959401 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959410 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959419 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959429 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959438 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959448 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.959457 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.959470 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.961950 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.961987 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962075 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962089 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962154 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962167 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962231 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962243 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962305 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962316 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962398 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962410 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962467 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962480 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962551 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962564 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962619 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962631 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962707 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962720 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962797 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962834 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962897 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962913 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.962979 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.962992 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963057 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963070 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963131 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963143 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963216 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963229 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963301 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963315 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963378 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963392 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963456 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963470 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963577 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963592 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963660 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963674 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963739 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963752 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963809 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963822 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963888 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963900 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.963956 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.963984 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964052 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964064 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964128 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964140 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964200 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964212 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964271 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964284 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964364 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964377 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964433 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964446 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964521 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964535 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964592 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964625 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964684 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964696 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964757 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964770 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964844 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964858 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.964947 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.964966 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965066 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965082 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965143 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965156 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965225 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965238 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965332 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965347 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965622 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965661 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965748 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965783 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965863 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.965899 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.965983 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966017 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966097 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966132 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966208 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966242 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966342 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966389 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966474 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966510 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966590 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966625 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966707 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966743 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966822 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966855 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.966945 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.966979 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967058 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967117 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967200 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967234 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967319 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967354 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967440 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967475 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967561 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967597 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967683 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967717 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967796 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967831 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.967915 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.967951 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968034 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968070 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968178 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968214 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968325 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968362 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968449 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968485 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968579 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968615 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968700 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968735 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968833 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968868 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.968955 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.968992 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969067 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969100 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969179 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969213 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969295 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969334 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969420 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969456 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969543 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969590 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969680 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969725 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969819 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.969864 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.969969 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970010 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970102 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970144 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970240 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970280 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970394 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970435 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970537 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970579 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970681 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970723 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970818 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.970859 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.970994 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971041 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971153 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971196 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971288 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971328 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971424 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971465 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971565 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971615 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971722 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971771 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.971875 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.971916 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972008 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972049 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972148 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972189 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972279 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972319 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972421 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972464 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972563 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972604 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972714 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972756 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972847 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.972889 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.972983 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.973023 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.973119 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.973160 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W1018 05:14:17.973935 31016 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1018 05:14:17.973980 31016 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1018 05:14:17.974008 31016 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1018 05:14:17.974034 31016 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1018 05:14:17.974070 31016 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1018 05:14:17.974103 31016 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1018 05:14:17.974128 31016 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1018 05:14:17.974184 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974222 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974250 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.974277 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974313 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.974348 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974377 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.974403 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974446 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.974487 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974516 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974544 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974587 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974615 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.974642 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.974677 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983233 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983268 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983337 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983361 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983431 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983448 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983511 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983527 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983625 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983643 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983706 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983732 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983815 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983833 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983900 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.983919 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.983982 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984000 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984067 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984086 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984157 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984175 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984256 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984274 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984352 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984371 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984453 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984486 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984557 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984580 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984646 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984671 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984740 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984759 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984821 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984844 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.984920 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.984955 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985045 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985065 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985146 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985164 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985239 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985262 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985327 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985345 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985412 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985430 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985522 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985541 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985634 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985653 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985714 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985732 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985830 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985849 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.985951 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.985969 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986038 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986060 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986121 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986139 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986208 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986226 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986291 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986310 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986377 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986396 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986472 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986489 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986558 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986575 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986643 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986661 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986724 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986747 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986807 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986824 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986901 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.986918 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.986987 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987004 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987084 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987102 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987170 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987187 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987258 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987276 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987340 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987358 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987422 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987436 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987498 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987515 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987578 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987595 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987662 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987679 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987741 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987759 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987817 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987835 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.987907 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.987930 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988023 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988047 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988114 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988135 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988192 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988210 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988282 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988302 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988363 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988384 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988464 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988481 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988538 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988555 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988632 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988648 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988712 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988729 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988791 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988809 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988877 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988895 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.988976 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.988993 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989084 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989103 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989166 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989188 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989253 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989269 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989334 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989352 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989424 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989443 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989503 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989527 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989594 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989611 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989674 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989692 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989755 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989771 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989917 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.989936 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.989996 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990013 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990090 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990108 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990214 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990232 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990296 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990314 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990372 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990396 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990462 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990480 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990553 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990572 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990632 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990649 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990712 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990729 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990803 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990821 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990901 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.990919 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.990978 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991000 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991062 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991079 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991152 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991170 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991234 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991258 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991339 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991357 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991415 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991433 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991520 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991544 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991616 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991635 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991696 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991722 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991782 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991800 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991868 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991889 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.991954 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.991973 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.992034 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.992051 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1018 05:14:17.992111 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1018 05:14:17.992129 31016 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.064s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.063s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.100s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.00s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.06s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.203s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.06s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.04s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.04s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.04s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.02s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.02s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.02s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.02s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.295s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I1018 05:14:25.517776 32794 files.go:57] Writing file "/tmp/TestWriteFile511071899/001/SubDir/test1.tmp" I1018 05:14:25.517972 32794 files.go:113] Changing file mode for "/tmp/TestWriteFile511071899/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.078s === RUN TestBootstrapChannelBuilder_BuildTasks I1018 05:14:31.061196 33497 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:31.061440 33497 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:31.061496 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:31.061540 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:31.061591 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:31.061642 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:31.061659 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:31.061679 33497 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:31.061708 33497 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:31.068983 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.127113 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.258327 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.318111 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.417593 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.511455 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.570449 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.630929 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:31.724930 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:32.190943 33497 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.21s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I1018 05:14:32.267117 33497 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:32.267312 33497 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:32.267399 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:32.267449 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:32.267502 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:32.267542 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:32.267572 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:32.267609 33497 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:32.267636 33497 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:32.267668 33497 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I1018 05:14:32.269524 33497 zones.go:93] ListHostedZonesPages { } I1018 05:14:32.360911 33497 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.09s) === RUN TestBootstrapChannelBuilder_AWSCloudController I1018 05:14:32.361195 33497 featureflag.go:160] FeatureFlag "ImageDigest"=false I1018 05:14:32.361430 33497 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:32.361510 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1018 05:14:32.361562 33497 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1018 05:14:32.361639 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1018 05:14:32.361684 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1018 05:14:32.361713 33497 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1018 05:14:32.361749 33497 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1018 05:14:32.361776 33497 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1018 05:14:32.363853 33497 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.11s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.05s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.14s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W1018 05:14:32.685163 33497 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.01s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I1018 05:14:32.739261 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739292 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739309 33497 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1018 05:14:32.739338 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1018 05:14:32.739353 33497 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I1018 05:14:32.739381 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1018 05:14:32.739407 33497 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I1018 05:14:32.739433 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739450 33497 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1018 05:14:32.739476 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739492 33497 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1018 05:14:32.739519 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739553 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739582 33497 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1018 05:14:32.739597 33497 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1018 05:14:32.739612 33497 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I1018 05:14:32.739626 33497 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I1018 05:14:32.739640 33497 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I1018 05:14:32.739656 33497 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1018 05:14:32.739672 33497 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I1018 05:14:32.739687 33497 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I1018 05:14:32.741045 33497 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.770s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I1018 05:14:28.510476 33340 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1525542090/001 I1018 05:14:28.514302 33340 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1525542090/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestAutoscalingGroupCloudformationRender I1018 05:14:28.515536 33340 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender2662716856/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I1018 05:14:28.516294 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1018 05:14:28.516400 33340 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1018 05:14:28.516517 33340 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I1018 05:14:28.516727 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.516839 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.516917 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.516948 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.517125 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.517169 33340 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1018 05:14:28.517222 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.517297 33340 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1018 05:14:28.517385 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1018 05:14:28.517538 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.517598 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.517644 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.517671 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.517721 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.517767 33340 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1018 05:14:28.517838 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I1018 05:14:28.518123 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.518184 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.518306 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.518361 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.518400 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.518439 33340 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1018 05:14:28.518552 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.518737 33340 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1018 05:14:28.518800 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.518840 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.518881 33340 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1018 05:14:28.518945 33340 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.519104 33340 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I1018 05:14:28.519184 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.519219 33340 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I1018 05:14:28.519271 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1018 05:14:28.519445 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.519505 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.519556 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.519582 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.519646 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.519681 33340 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1018 05:14:28.519731 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.519799 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.519836 33340 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1018 05:14:28.519882 33340 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.519934 33340 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I1018 05:14:28.519977 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1018 05:14:28.520031 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I1018 05:14:28.520194 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1018 05:14:28.520245 33340 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1018 05:14:28.520280 33340 internetgateways.go:66] CreateInternetGateway: { } I1018 05:14:28.520314 33340 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I1018 05:14:28.520446 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.520505 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.520550 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.520576 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.520658 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.520690 33340 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1018 05:14:28.520739 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.520805 33340 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1018 05:14:28.520898 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1018 05:14:28.521024 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.521075 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.521121 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.521147 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.521197 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.521243 33340 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1018 05:14:28.521323 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I1018 05:14:28.522171 33340 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender970186574/001 I1018 05:14:28.522459 33340 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender970186574/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I1018 05:14:28.524488 33340 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender579338744/001 I1018 05:14:28.525988 33340 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender579338744/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I1018 05:14:28.526647 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.526716 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.526831 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.526896 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.526936 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.526975 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.527056 33340 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1018 05:14:28.527258 33340 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I1018 05:14:28.527308 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1018 05:14:28.527356 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1018 05:14:28.527490 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.527546 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.527596 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.527623 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.527665 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.527701 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.527752 33340 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1018 05:14:28.527833 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I1018 05:14:28.528254 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.528329 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.528431 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.528476 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.528510 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.528547 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.528598 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.528707 33340 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1018 05:14:28.528756 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.528790 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.528831 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1018 05:14:28.528964 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.529018 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.529067 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.529094 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.529135 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.529169 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.529217 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.529282 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.529315 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I1018 05:14:28.529599 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.529655 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.529745 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.529788 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.529822 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.529862 33340 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1018 05:14:28.529921 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.530027 33340 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1018 05:14:28.530001 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.530161 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.530198 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.530243 33340 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:14:28.530290 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1018 05:14:28.530426 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.530480 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.530531 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.530559 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.530600 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.530635 33340 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1018 05:14:28.530683 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.530753 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.530779 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.530842 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I1018 05:14:28.531105 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.531158 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.531250 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.531295 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.531328 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.531365 33340 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1018 05:14:28.531411 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.531492 33340 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1018 05:14:28.531520 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.531706 33340 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1018 05:14:28.531755 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.531781 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.531807 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1018 05:14:28.531958 33340 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1018 05:14:28.532003 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.532046 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.532062 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.532087 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.532110 33340 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1018 05:14:28.532145 33340 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1018 05:14:28.532241 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.532271 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.532317 33340 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I1018 05:14:28.532463 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I1018 05:14:28.532527 33340 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I1018 05:14:28.532652 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.532694 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.532731 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.532746 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.532808 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.532834 33340 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1018 05:14:28.532865 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.532898 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.533001 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1018 05:14:28.533028 33340 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1018 05:14:28.533057 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1018 05:14:28.533183 33340 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1018 05:14:28.533223 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.533260 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.533275 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.533308 33340 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1018 05:14:28.533347 33340 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1018 05:14:28.533398 33340 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I1018 05:14:28.533624 33340 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.533664 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.533761 33340 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1018 05:14:28.533809 33340 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1018 05:14:28.533839 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.533861 33340 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.533964 33340 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.534016 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1018 05:14:28.534064 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.534079 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.534102 33340 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1018 05:14:28.534126 33340 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I1018 05:14:28.534299 33340 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I1018 05:14:28.534415 33340 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.534456 33340 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I1018 05:14:28.534497 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1018 05:14:28.534511 33340 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1018 05:14:28.534540 33340 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.071s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.043s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.099s === RUN TestDiskRenderAzure I1018 05:14:24.944467 32618 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I1018 05:14:24.944986 32618 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I1018 05:14:24.945620 32618 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I1018 05:14:24.946351 32618 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I1018 05:14:24.946928 32618 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I1018 05:14:24.947392 32618 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I1018 05:14:24.947948 32618 resourcegroup.go:103] Creating a new Resource Group with name: rg I1018 05:14:24.947984 32618 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I1018 05:14:24.948407 32618 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I1018 05:14:24.949767 32618 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I1018 05:14:24.950419 32618 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I1018 05:14:24.950912 32618 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I1018 05:14:24.951734 32618 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I1018 05:14:24.952195 32618 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.076s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.119s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I1018 05:14:28.338341 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.338815 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.338932 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.339347 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.339453 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.339586 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I1018 05:14:28.339767 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.339841 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.339937 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.340030 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.340126 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.340187 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I1018 05:14:28.340349 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.340531 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.340626 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.340910 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1018 05:14:28.341014 33324 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1018 05:14:28.341136 33324 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.051s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.01s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.044s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.029s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.01s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.041s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.033s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.060s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I1018 05:14:30.861084 33481 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I1018 05:14:30.861142 33481 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I1018 05:14:30.861154 33481 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I1018 05:14:30.861169 33481 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I1018 05:14:30.861179 33481 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I1018 05:14:30.861191 33481 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I1018 05:14:30.861202 33481 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W1018 05:14:30.861671 33481 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.060s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.096s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.037s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.025s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.004s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W1018 05:14:27.106478 33203 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.023s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.019s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.004s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.004s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.022s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W1018 05:14:31.918437 33611 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.064s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 129.0 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.1-r0.apk >>> kops: Build complete at Tue, 18 Oct 2022 05:15:28 +0000 elapsed time 0h 3m 28s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20221018.051201) (2/3) Purging kubernetes (1.25.0-r2) (3/3) Purging go (1.19.2-r0) Executing busybox-1.35.0-r27.trigger OK: 393 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...