>>> kops: Building testing/kops 1.26.3-r0 (using abuild 3.11.2-r0) started Sun, 28 May 2023 04:34:09 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go bash (1/5) Installing kubernetes (1.27.2-r0) (2/5) Installing go (1.20.4-r0) (3/5) Installing readline (8.2.1-r2) (4/5) Installing bash (5.2.15-r5) Executing bash-5.2.15-r5.post-install (5/5) Installing .makedepends-kops (20230528.043411) Executing busybox-1.36.1-r0.trigger OK: 547 MiB in 108 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.26.3.tar.gz::https://github.com/kubernetes/kops/archive/v1.26.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 9408k 0 9408k 0 0 6209k 0 --:--:-- 0:00:01 --:--:-- 10.3M 100 14.2M 0 14.2M 0 0 5781k 0 --:--:-- 0:00:02 --:--:-- 7700k 100 19.3M 0 19.3M 0 0 5634k 0 --:--:-- 0:00:03 --:--:-- 6861k 100 23.9M 0 23.9M 0 0 5396k 0 --:--:-- 0:00:04 --:--:-- 6265k 100 28.4M 0 28.4M 0 0 5272k 0 --:--:-- 0:00:05 --:--:-- 5949k 100 31.3M 0 31.3M 0 0 5177k 0 --:--:-- 0:00:06 --:--:-- 4843k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz >>> kops: Checking sha512sums... kops-1.26.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.26.3.tar.gz... go: downloading github.com/aws/aws-sdk-go v1.44.217 go: downloading k8s.io/klog/v2 v2.90.1 go: downloading google.golang.org/api v0.111.0 go: downloading k8s.io/apimachinery v0.26.2 go: downloading k8s.io/api v0.26.2 go: downloading github.com/gophercloud/gophercloud v1.2.0 go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/cert-manager/cert-manager v1.11.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.1 go: downloading github.com/google/go-containerregistry v0.13.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading golang.org/x/crypto v0.7.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/spf13/viper v1.15.0 go: downloading go.uber.org/multierr v1.10.0 go: downloading helm.sh/helm/v3 v3.11.2 go: downloading k8s.io/cli-runtime v0.26.2 go: downloading k8s.io/client-go v0.26.2 go: downloading github.com/stretchr/testify v1.8.2 go: downloading sigs.k8s.io/controller-runtime v0.14.5 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading cloud.google.com/go/compute v1.18.0 go: downloading golang.org/x/oauth2 v0.6.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/kubectl v0.26.2 go: downloading k8s.io/cloud-provider-gcp/providers v0.26.2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/imdario/mergo v0.3.13 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading golang.org/x/sync v0.1.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading k8s.io/utils v0.0.0-20230308161112-d77c459e9343 go: downloading github.com/pkg/errors v0.9.1 go: downloading golang.org/x/net v0.8.0 go: downloading golang.org/x/term v0.6.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading k8s.io/component-base v0.26.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading k8s.io/apiserver v0.26.2 go: downloading github.com/Masterminds/sprig/v3 v3.2.3 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/Azure/azure-sdk-for-go v68.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/digitalocean/godo v1.97.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading k8s.io/cloud-provider-aws v1.26.1 go: downloading github.com/hetznercloud/hcloud-go v1.41.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/onsi/ginkgo/v2 v2.6.1 go: downloading github.com/onsi/gomega v1.24.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/scaleway/scaleway-sdk-go v1.0.0-beta.14 go: downloading google.golang.org/appengine v1.6.7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.26.2 go: downloading k8s.io/mount-utils v0.26.2 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading k8s.io/kubelet v0.26.2 go: downloading golang.org/x/sys v0.6.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading github.com/sergi/go-diff v1.3.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.334.0 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/google/go-tpm-tools v0.3.10 go: downloading github.com/google/go-tpm v0.3.3 go: downloading golang.org/x/exp v0.0.0-20230307190834-24139beb5833 go: downloading k8s.io/gengo v0.0.0-20230306165830-ab3349d207d4 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/docker/cli v20.10.21+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading golang.org/x/text v0.8.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/subosito/gotenv v1.4.2 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.7 go: downloading github.com/pelletier/go-toml/v2 v2.0.6 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading golang.org/x/time v0.3.0 go: downloading k8s.io/kube-openapi v0.0.0-20221207184640-f3cff1453715 go: downloading k8s.io/apiextensions-apiserver v0.26.1 go: downloading sigs.k8s.io/gateway-api v0.6.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.39.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.2.0 go: downloading github.com/huandu/xstrings v1.4.0 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.3.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading google.golang.org/grpc v1.53.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/google/go-querystring v1.1.0 go: downloading k8s.io/csi-translation-lib v0.26.0 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.24.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/moby/sys/mountinfo v0.6.2 go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/kr/pretty v0.3.1 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/containerd/containerd v1.6.15 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.2 go: downloading github.com/distribution/distribution/v3 v3.0.0-20221208165359-362910506bc2 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/emicklei/go-restful/v3 v3.10.1 go: downloading google.golang.org/genproto v0.0.0-20230223222841-637eb2293923 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/google/go-sev-guest v0.4.1 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/dnaeon/go-vcr v1.2.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading go.uber.org/atomic v1.10.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.18 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.9.0 go: downloading github.com/docker/docker v20.10.21+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.3 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/go-openapi/jsonreference v0.20.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/google/logger v1.1.1 go: downloading github.com/pborman/uuid v1.2.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.3 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/josharian/intern v1.0.0 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/Microsoft/hcsshim v0.9.6 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading golang.org/x/mod v0.8.0 go: downloading github.com/containerd/cgroups v1.0.4 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.3.0 go: downloading cloud.google.com/go v0.107.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version=1.26.3 -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0528 04:36:36.313850 25077 addon.go:233] addon test wants to update all nodes I0528 04:36:36.316087 25077 addon.go:233] addon test wants to update workers nodes I0528 04:36:36.316598 25077 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0528 04:36:36.317289 25077 addon.go:266] installing PKI for "test" I0528 04:36:37.057291 25077 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (0.95s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.020s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.093s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kopscontrollerclient [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/model/scalewaymodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/nodeidentity/scaleway [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/scaleway [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/scaleway [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scaleway [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scalewaytasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/install [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestCreateClusterMinimal I0528 04:36:42.449423 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.449662 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.449713 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.449777 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.449820 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.449888 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.449950 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.449977 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.450022 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.450066 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.456257 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.456317 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.548940 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:42.549160 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:42.549582 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:42.563531 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.563617 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.563644 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.563676 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.563700 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.563742 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.563770 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.563793 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.563817 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.563835 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.568295 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.568357 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.604873 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:42.605180 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:42.605677 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:42.614138 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.614245 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.614282 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.614331 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.614383 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.614430 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.614470 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.614504 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.614541 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.614569 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.618919 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.618972 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.711620 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:42.711790 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:42.712117 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:42.718146 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.718271 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.718312 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.718357 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.718402 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.718455 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.718500 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.718536 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.718579 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.718610 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.750056 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.750115 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.817323 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:42.817575 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:42.817942 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:42.823383 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.823471 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.823504 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.823552 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.823589 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.823638 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.823676 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.823709 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.823747 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.823775 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.827621 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.827717 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.926850 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:42.927026 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:42.927335 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:42.932889 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:42.932977 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:42.933003 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:42.933042 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:42.933068 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:42.933100 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:42.933126 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:42.933142 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:42.933164 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:42.933179 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:42.936634 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:42.936690 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.014549 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.015506 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.016983 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:43.026618 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.026724 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.026763 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.026812 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.026855 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.026900 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.026967 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.027009 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.027050 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.027082 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.031623 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.031668 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.100238 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.100420 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.100712 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:43.106109 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.106181 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.106204 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.106233 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.106258 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.106287 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.106312 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.106328 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.106350 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.106366 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.110783 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.110829 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.174386 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.174488 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.174807 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:43.183047 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.183128 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.183157 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.183190 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.183218 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.183251 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.183276 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.183292 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.183317 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.183335 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.215399 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.215515 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.252793 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.252988 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.253318 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.81s) === RUN TestCreateClusterHetzner I0528 04:36:43.259314 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.259428 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.259460 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.259507 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.259544 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.259592 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.259629 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.259660 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.259699 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.259726 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.263347 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.263386 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:43.357122 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:43.357167 26677 new_cluster.go:1354] Cloud Provider ID: "hetzner" I0528 04:36:43.365508 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.365723 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.365770 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.366417 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.366479 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.366519 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.366546 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.366564 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.366590 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.366609 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.370493 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.370534 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.396653 26677 new_cluster.go:1354] Cloud Provider ID: "hetzner" --- PASS: TestCreateClusterHetzner (0.14s) === RUN TestCreateClusterOpenStack I0528 04:36:43.401935 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.402024 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.402053 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.402082 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.402107 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.402137 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.402163 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.402180 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.402202 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.402218 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.406322 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.406377 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:43.454805 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:43.454850 26677 new_cluster.go:1354] Cloud Provider ID: "openstack" I0528 04:36:43.457672 26677 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 --- PASS: TestCreateClusterOpenStack (0.06s) === RUN TestCreateClusterOpenStackOctavia I0528 04:36:43.465206 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.465292 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.465319 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.465349 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.465372 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.465402 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.465427 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.465443 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.465469 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.465487 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.469080 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.469122 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:43.503804 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:43.503861 26677 new_cluster.go:1354] Cloud Provider ID: "openstack" I0528 04:36:43.506422 26677 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0528 04:36:43.506452 26677 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackOctavia (0.06s) === RUN TestCreateClusterOpenStackNoDNS I0528 04:36:43.522126 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.522281 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.522324 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.522376 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.522418 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.522474 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.522521 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.522557 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.522597 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.522631 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.528025 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.528080 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:43.557072 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:43.557117 26677 new_cluster.go:1354] Cloud Provider ID: "openstack" I0528 04:36:43.557167 26677 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0528 04:36:43.557180 26677 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackNoDNS (0.04s) === RUN TestCreateClusterCilium I0528 04:36:43.565108 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.565198 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.565224 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.565252 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.565275 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.565303 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.565330 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.565346 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.565369 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.565384 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.569132 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.569181 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.634086 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.634261 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.634626 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterCilium (0.08s) === RUN TestCreateClusterOverride I0528 04:36:43.640378 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.640463 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.640488 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.640516 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.640540 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.640572 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.640597 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.640613 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.640636 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.640653 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.644542 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.644587 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.692169 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.692673 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.692972 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterKubernetesFeatureGates I0528 04:36:43.698168 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.698246 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.698283 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.698316 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.698341 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.698371 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.698397 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.698413 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.698436 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.698453 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.702638 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.702703 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.751981 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.752155 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.752526 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterKubernetesFeatureGates (0.06s) === RUN TestCreateClusterComplex I0528 04:36:43.759174 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.759253 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.759275 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.759302 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.759327 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.759359 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.759386 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.759404 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.759429 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.759448 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.794696 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.794740 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.848184 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.848365 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.848655 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.09s) === RUN TestCreateClusterHA I0528 04:36:43.853767 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.853849 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.853873 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.853899 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.853921 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.853950 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.853976 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.853993 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.854016 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.854032 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.857915 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.857973 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.894593 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.895022 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.895040 26677 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0528 04:36:43.895051 26677 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0528 04:36:43.895378 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:43.903667 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.903742 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.903766 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.903794 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.903816 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.903846 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.903871 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.903886 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.903908 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.903925 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.907665 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.907706 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.947164 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:43.947490 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:43.947507 26677 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0528 04:36:43.947517 26677 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0528 04:36:43.947839 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.10s) === RUN TestCreateClusterMinimalGCE I0528 04:36:43.956091 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.956173 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.956198 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.956224 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.956247 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.956289 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.956319 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.956336 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.956358 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.956375 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.960342 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.960388 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.989347 26677 new_cluster.go:1354] Cloud Provider ID: "gce" I0528 04:36:43.989462 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 I0528 04:36:43.995007 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:43.995073 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:43.995098 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:43.995140 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:43.995168 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:43.995197 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:43.995223 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:43.995241 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:43.995264 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:43.995281 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:43.998857 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:43.998898 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.066734 26677 new_cluster.go:1354] Cloud Provider ID: "gce" I0528 04:36:44.066819 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterMinimalGCE (0.12s) === RUN TestCreateClusterHAGCE I0528 04:36:44.072240 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.072330 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.072358 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.072391 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.072415 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.072457 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.072484 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.072500 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.072526 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.072544 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.076299 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.076340 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.133716 26677 new_cluster.go:1354] Cloud Provider ID: "gce" I0528 04:36:44.133834 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.07s) === RUN TestCreateClusterGCE I0528 04:36:44.142675 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.142768 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.142796 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.142827 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.142854 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.142884 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.142910 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.142930 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.142958 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.142976 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.147775 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.147841 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.205047 26677 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0528 04:36:44.205121 26677 new_cluster.go:1354] Cloud Provider ID: "gce" I0528 04:36:44.205191 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.07s) === RUN TestCreateClusterHASharedZone I0528 04:36:44.211517 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.211645 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.211677 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.211710 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.211739 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.211776 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.211805 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.211829 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.211856 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.211876 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.217041 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.217115 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:44.278915 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:44.278959 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.280284 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:44.281737 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.08s) === RUN TestCreateClusterHASharedZones I0528 04:36:44.290107 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.290220 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.290246 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.290277 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.290302 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.290332 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.290357 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.290372 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.290396 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.290413 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.295384 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.295429 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0528 04:36:44.337386 26677 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0528 04:36:44.337434 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.337796 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:44.337811 26677 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0528 04:36:44.338158 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.06s) === RUN TestCreateClusterPrivate I0528 04:36:44.347797 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.347907 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.347943 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.347979 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.348005 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.348037 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.348064 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.348082 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.348110 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.348127 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.353456 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.353499 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.419995 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.420263 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:44.420282 26677 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0528 04:36:44.420643 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:44.420675 26677 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.08s) === RUN TestCreateClusterPrivateGCE I0528 04:36:44.427238 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.427329 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.427356 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.427387 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.427411 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.427442 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.427469 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.427487 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.427515 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.427533 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.432285 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.432324 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.483667 26677 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0528 04:36:44.483782 26677 new_cluster.go:1354] Cloud Provider ID: "gce" I0528 04:36:44.483914 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.07s) === RUN TestCreateClusterWithNGWSpecified I0528 04:36:44.494776 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.494903 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.494955 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.495010 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.495058 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.495104 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.495145 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.495177 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.495216 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.495244 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.500295 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.500342 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.562645 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.562868 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:44.562884 26677 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0528 04:36:44.563216 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:44.563246 26677 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.07s) === RUN TestCreateClusterWithINGWSpecified I0528 04:36:44.569481 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.569574 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.569600 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.569631 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.569655 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.569685 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.569711 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.569727 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.569752 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.569769 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.574465 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.574534 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.685795 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.686016 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:44.686033 26677 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0528 04:36:44.686374 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:44.686404 26677 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.12s) === RUN TestCreateClusterSharedVPC I0528 04:36:44.692692 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.692783 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.692809 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.692840 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.692868 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.692900 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.692926 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.692942 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.692966 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.692984 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.698654 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.698733 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.727445 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.727916 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.727994 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.728070 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.728111 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.728209 26677 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0528 04:36:44.728781 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.04s) === RUN TestCreateClusterSharedSubnets I0528 04:36:44.735685 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.735777 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.735803 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.735834 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.735862 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.735893 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.735919 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.735934 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.735958 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.735975 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.740005 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.740043 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.822726 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.822775 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.822863 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.823006 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.823026 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.823048 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.823063 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.823381 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.09s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0528 04:36:44.828781 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.828870 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.828894 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.828923 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.828947 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.828978 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.829005 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.829022 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.829047 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.829063 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.832904 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.832944 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.887040 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:36:44.887109 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.887147 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.887268 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.887513 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.887547 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.887589 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.887616 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.888113 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.07s) === RUN TestCreateClusterPrivateSharedSubnets I0528 04:36:44.897230 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.897366 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.897406 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.897451 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.897492 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.897543 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.897588 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.897619 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.897664 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.897696 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.902779 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.902834 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.956934 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.956987 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.957043 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.957060 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.957118 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:44.957264 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.957284 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.957305 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:36:44.957319 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:36:44.957661 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.07s) === RUN TestCreateClusterIPv6 I0528 04:36:44.965051 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:44.965263 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:44.965316 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:44.965407 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:44.965463 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:44.965527 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:44.965808 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:44.967027 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:44.967095 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:44.967129 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:44.973302 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:44.973358 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:45.003275 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:45.003451 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet dualstack-us-test-1a I0528 04:36:45.003469 26677 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0528 04:36:45.004021 26677 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestCreateClusterDifferentAMIs I0528 04:36:45.010438 26677 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0528 04:36:45.010451 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:45.010531 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:45.010557 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:45.010589 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:45.010612 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:45.010642 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:45.010668 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:45.010685 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:45.010710 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:45.010728 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:45.016249 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:45.016341 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:45.058494 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:45.058588 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:45.058603 26677 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0528 04:36:45.058947 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:45.066530 26677 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.06s) === RUN TestCreateClusterKarpenter I0528 04:36:45.066631 26677 featureflag.go:166] FeatureFlag "Karpenter"=true I0528 04:36:45.066649 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:45.066750 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:45.066776 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:45.066804 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:45.066826 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:45.066857 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:45.066883 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:45.066899 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:45.066922 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:45.066938 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:45.070409 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:45.070449 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0528 04:36:45.137864 26677 new_cluster.go:1354] Cloud Provider ID: "aws" I0528 04:36:45.138043 26677 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0528 04:36:45.138059 26677 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0528 04:36:45.138069 26677 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0528 04:36:45.138440 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:45.144416 26677 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.08s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0528 04:36:45.144661 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:45.144735 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:45.144759 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:45.144819 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:45.144843 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:45.144872 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:45.144896 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:45.144913 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:45.144948 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:45.144964 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:45.150055 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0528 04:36:45.151147 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:45.151218 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:45.151238 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:45.151258 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:45.151278 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:45.151303 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:45.151350 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:45.151367 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:45.151386 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:45.151403 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:45.173039 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:36:45.176103 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:45.176216 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:36:48.337650 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.978262 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:36:50.980772 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:50.980803 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:50.980904 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:50.982571 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:36:50.987376 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:36:50.988860 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:50.989099 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:50.990709 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:50.991304 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:50.991418 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:36:50.992093 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:36:50.992464 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:36:50.996717 26677 target.go:116] Terraform output is in /tmp/test3513763521/out --- PASS: TestMinimal (5.85s) === RUN TestMinimal_v1_23 I0528 04:36:50.999816 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.999944 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.999986 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:51.000035 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:51.000072 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:51.000138 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:51.000222 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:51.000252 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:51.000288 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:51.000316 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:51.025833 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0528 04:36:51.028987 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:51.029081 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:52.831016 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:52.978869 26677 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0528 04:36:52.979431 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:52.979472 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:52.979571 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:52.985357 26677 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0528 04:36:52.990837 26677 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0528 04:36:52.991174 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:52.991385 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:52.992439 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:52.992984 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:52.993051 26677 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0528 04:36:52.993245 26677 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0528 04:36:52.993366 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0528 04:36:52.996617 26677 target.go:116] Terraform output is in /tmp/test230379809/out --- PASS: TestMinimal_v1_23 (2.00s) === RUN TestMinimal_v1_24 I0528 04:36:52.998826 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.998909 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.998935 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.998963 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.998989 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.999023 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.999073 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.999092 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.999116 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.999134 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:53.014890 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:36:53.016795 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:53.016856 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.332251 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:54.447218 26677 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0528 04:36:54.447537 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:54.447561 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:54.447646 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:54.453175 26677 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0528 04:36:54.456398 26677 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0528 04:36:54.456567 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.456808 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.458188 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.458801 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.458870 26677 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0528 04:36:54.459099 26677 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0528 04:36:54.459231 26677 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0528 04:36:54.462553 26677 target.go:116] Terraform output is in /tmp/test3869034174/out --- PASS: TestMinimal_v1_24 (1.47s) === RUN TestMinimal_v1_25 I0528 04:36:54.464948 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:54.465024 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:54.465048 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:54.465077 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:54.465100 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:54.465133 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:54.465183 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:54.465202 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:54.465224 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:54.465241 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:54.481544 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:36:54.483787 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:54.483846 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:55.912922 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:56.052728 26677 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0528 04:36:56.053402 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:56.053431 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:56.053523 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:56.054744 26677 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0528 04:36:56.057954 26677 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0528 04:36:56.058196 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:56.058378 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:56.059159 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:56.059969 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:56.060039 26677 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0528 04:36:56.060241 26677 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0528 04:36:56.060365 26677 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0528 04:36:56.063180 26677 target.go:116] Terraform output is in /tmp/test3502963444/out --- PASS: TestMinimal_v1_25 (1.60s) === RUN TestMinimal_v1_26 I0528 04:36:56.065458 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:56.065541 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:56.065567 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:56.065594 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:56.065617 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:56.065649 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:56.065701 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:56.065718 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:56.065743 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:56.065760 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:56.081566 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:36:56.083483 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:56.083546 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.721201 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:57.833625 26677 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0528 04:36:57.834134 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:57.834168 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:57.834263 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:57.835830 26677 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0528 04:36:57.838829 26677 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0528 04:36:57.839026 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.839223 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.840076 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.842247 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.842400 26677 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0528 04:36:57.842746 26677 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0528 04:36:57.842926 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0528 04:36:57.847432 26677 target.go:116] Terraform output is in /tmp/test1647424259/out --- PASS: TestMinimal_v1_26 (1.79s) === RUN TestMinimal_NoneDNS I0528 04:36:57.851181 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:57.851295 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:57.851333 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:57.851376 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:57.851420 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:57.851470 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:57.851559 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:57.851605 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:57.851648 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:57.851681 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:36:57.885284 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:57.885390 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:58.052138 26677 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0528 04:36:58.057498 26677 executor.go:111] Tasks: 43 done / 98 total; 20 can run I0528 04:36:58.059526 26677 executor.go:111] Tasks: 63 done / 98 total; 25 can run I0528 04:36:58.060471 26677 executor.go:111] Tasks: 88 done / 98 total; 2 can run I0528 04:36:58.060558 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:36:58.060634 26677 loadbalancers.go:35] PageSize not implemented I0528 04:36:58.060736 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:36:58.060803 26677 loadbalancers.go:35] PageSize not implemented I0528 04:36:58.065297 26677 executor.go:111] Tasks: 90 done / 98 total; 4 can run I0528 04:36:58.065535 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:58.065757 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:58.066798 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:58.068437 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:58.068549 26677 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0528 04:36:58.068822 26677 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0528 04:36:58.068946 26677 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0528 04:36:58.073708 26677 target.go:116] Terraform output is in /tmp/test903083432/out --- PASS: TestMinimal_NoneDNS (0.23s) === RUN TestHetzner I0528 04:36:58.076366 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:36:59.444385 26677 executor.go:111] Tasks: 0 done / 46 total; 37 can run I0528 04:36:59.450430 26677 executor.go:111] Tasks: 37 done / 46 total; 3 can run I0528 04:36:59.450677 26677 executor.go:111] Tasks: 40 done / 46 total; 2 can run I0528 04:36:59.454164 26677 executor.go:111] Tasks: 42 done / 46 total; 4 can run I0528 04:36:59.455996 26677 executor.go:111] Tasks: 46 done / 46 total; 0 can run I0528 04:36:59.457373 26677 target.go:116] Terraform output is in /tmp/test1104362711/out --- PASS: TestHetzner (1.38s) === RUN TestNvidia I0528 04:36:59.459157 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:59.459237 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:59.459265 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:59.459298 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:59.459323 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:59.459355 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:59.459409 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:59.459426 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:59.459448 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:59.459465 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:59.475012 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:36:59.476858 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.476916 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.517345 26677 zones.go:93] ListHostedZonesPages { } I0528 04:36:59.635716 26677 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0528 04:36:59.636449 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:36:59.636485 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:36:59.636580 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:36:59.638273 26677 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0528 04:36:59.641503 26677 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0528 04:36:59.641875 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.642130 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.643358 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.643672 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.643740 26677 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0528 04:36:59.643997 26677 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0528 04:36:59.644114 26677 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0528 04:36:59.647788 26677 target.go:116] Terraform output is in /tmp/test2993740169/out --- PASS: TestNvidia (0.19s) === RUN TestMinimalGossip I0528 04:36:59.651108 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:59.651212 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:59.651254 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:59.651304 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:59.651357 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:59.651412 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:59.651494 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:59.651526 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:59.651558 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:59.651577 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:36:59.670768 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.670856 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.771885 26677 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0528 04:36:59.777112 26677 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0528 04:36:59.781678 26677 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0528 04:36:59.781882 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.782035 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.782672 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.784457 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.784549 26677 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0528 04:36:59.784773 26677 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0528 04:36:59.784897 26677 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0528 04:36:59.789706 26677 target.go:116] Terraform output is in /tmp/test1104383528/out --- PASS: TestMinimalGossip (0.14s) === RUN TestMinimalGossipIRSA I0528 04:36:59.793284 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:59.793423 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:59.793467 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:59.793517 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:59.793565 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:59.793624 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:59.793715 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:59.793751 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:59.793793 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:59.793823 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:36:59.817536 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.817629 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.920934 26677 executor.go:111] Tasks: 0 done / 89 total; 44 can run I0528 04:36:59.924056 26677 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0528 04:36:59.931607 26677 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0528 04:36:59.932106 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.932312 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.932925 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.933865 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:36:59.933934 26677 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0528 04:36:59.934217 26677 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0528 04:36:59.934345 26677 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0528 04:36:59.938401 26677 target.go:116] Terraform output is in /tmp/test3268091460/out --- PASS: TestMinimalGossipIRSA (0.15s) === RUN TestMinimalGCE I0528 04:36:59.941520 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:59.943212 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:01.703976 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:01.704043 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:01.704124 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0528 04:37:01.704146 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0528 04:37:01.704161 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0528 04:37:01.704227 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:01.705199 26677 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0528 04:37:01.710899 26677 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0528 04:37:01.713438 26677 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0528 04:37:01.714749 26677 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0528 04:37:01.714902 26677 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0528 04:37:01.716702 26677 target.go:116] Terraform output is in /tmp/test2911507245/out --- PASS: TestMinimalGCE (1.78s) === RUN TestMinimalGCEPrivate I0528 04:37:01.718648 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:01.719698 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:01.895432 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:01.895498 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:01.895581 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0528 04:37:01.895601 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0528 04:37:01.895615 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0528 04:37:01.895663 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:01.896550 26677 executor.go:111] Tasks: 0 done / 65 total; 41 can run I0528 04:37:01.898653 26677 executor.go:111] Tasks: 41 done / 65 total; 17 can run I0528 04:37:01.901013 26677 executor.go:111] Tasks: 58 done / 65 total; 5 can run I0528 04:37:01.902694 26677 executor.go:111] Tasks: 63 done / 65 total; 2 can run I0528 04:37:01.902803 26677 executor.go:111] Tasks: 65 done / 65 total; 0 can run I0528 04:37:01.904414 26677 target.go:116] Terraform output is in /tmp/test1974183493/out --- PASS: TestMinimalGCEPrivate (0.19s) === RUN TestMinimalGCEInternalLoadBalancer I0528 04:37:01.905971 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:01.906922 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:02.071302 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:02.071358 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:02.071445 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0528 04:37:02.071465 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0528 04:37:02.071480 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0528 04:37:02.071532 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0528 04:37:02.071561 26677 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0528 04:37:02.072525 26677 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0528 04:37:02.074490 26677 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0528 04:37:02.077391 26677 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0528 04:37:02.077409 26677 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0528 04:37:02.079068 26677 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0528 04:37:02.079182 26677 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0528 04:37:02.079352 26677 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0528 04:37:02.079467 26677 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0528 04:37:02.081053 26677 target.go:116] Terraform output is in /tmp/test4289422844/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.18s) === RUN TestMinimalGCEPublicLoadBalancer I0528 04:37:02.082593 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.083578 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:02.288417 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:02.288477 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:02.288559 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/main" W0528 04:37:02.288576 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/events" W0528 04:37:02.288586 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com" W0528 04:37:02.288629 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:02.289469 26677 executor.go:111] Tasks: 0 done / 70 total; 43 can run I0528 04:37:02.295136 26677 executor.go:111] Tasks: 43 done / 70 total; 18 can run I0528 04:37:02.297636 26677 executor.go:111] Tasks: 61 done / 70 total; 7 can run I0528 04:37:02.300352 26677 executor.go:111] Tasks: 68 done / 70 total; 2 can run I0528 04:37:02.300581 26677 executor.go:111] Tasks: 70 done / 70 total; 0 can run I0528 04:37:02.303689 26677 target.go:116] Terraform output is in /tmp/test3540925872/out --- PASS: TestMinimalGCEPublicLoadBalancer (0.22s) === RUN TestMinimalGCELongClusterName I0528 04:37:02.306473 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.308069 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:02.481372 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:02.481445 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:02.481515 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0528 04:37:02.481532 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0528 04:37:02.481544 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0528 04:37:02.481598 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:02.484319 26677 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0528 04:37:02.487991 26677 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0528 04:37:02.490475 26677 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0528 04:37:02.491876 26677 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0528 04:37:02.491983 26677 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0528 04:37:02.493609 26677 target.go:116] Terraform output is in /tmp/test2844894865/out --- PASS: TestMinimalGCELongClusterName (0.19s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0528 04:37:02.495299 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.496250 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:02.656629 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:02.656681 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:02.656772 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0528 04:37:02.656788 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0528 04:37:02.656800 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0528 04:37:02.656842 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0528 04:37:02.656870 26677 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0528 04:37:02.657616 26677 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0528 04:37:02.659771 26677 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0528 04:37:02.662140 26677 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0528 04:37:02.662168 26677 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0528 04:37:02.663418 26677 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0528 04:37:02.663538 26677 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0528 04:37:02.663618 26677 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0528 04:37:02.663683 26677 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0528 04:37:02.665344 26677 target.go:116] Terraform output is in /tmp/test258909807/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.17s) === RUN TestMinimalGCEDNSNone I0528 04:37:02.666920 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.667803 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:02.813388 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:02.813443 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:02.813507 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0528 04:37:02.813522 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0528 04:37:02.813533 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0528 04:37:02.813570 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0528 04:37:02.813593 26677 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0528 04:37:02.814325 26677 executor.go:111] Tasks: 0 done / 67 total; 41 can run I0528 04:37:02.819791 26677 executor.go:111] Tasks: 41 done / 67 total; 16 can run W0528 04:37:02.822860 26677 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0528 04:37:02.822886 26677 executor.go:111] Tasks: 57 done / 67 total; 5 can run I0528 04:37:02.824509 26677 executor.go:111] Tasks: 62 done / 67 total; 2 can run I0528 04:37:02.824642 26677 executor.go:111] Tasks: 64 done / 67 total; 1 can run I0528 04:37:02.824738 26677 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0528 04:37:02.824895 26677 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0528 04:37:02.826796 26677 target.go:116] Terraform output is in /tmp/test2063543564/out --- PASS: TestMinimalGCEDNSNone (0.16s) === RUN TestHA I0528 04:37:02.829150 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.829231 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:02.829266 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:02.829306 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:02.829332 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:02.829363 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:02.829423 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:02.829439 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:02.829469 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:02.829486 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:02.848425 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:02.850473 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.850535 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.850575 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.850605 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.888293 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:02.956436 26677 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0528 04:37:02.956999 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:02.957021 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:02.957077 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:02.958612 26677 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0528 04:37:02.961668 26677 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0528 04:37:02.961814 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.962089 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.962250 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.962455 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.962594 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.963150 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.963685 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.964982 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:02.965093 26677 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0528 04:37:02.965394 26677 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0528 04:37:02.965548 26677 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0528 04:37:02.971860 26677 target.go:116] Terraform output is in /tmp/test2297820975/out --- PASS: TestHA (0.15s) === RUN TestHighAvailabilityGCE I0528 04:37:02.976041 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:02.978211 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:03.206322 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:03.206435 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:03.206561 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0528 04:37:03.206594 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0528 04:37:03.206619 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0528 04:37:03.206700 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0528 04:37:03.206785 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0528 04:37:03.206848 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:03.208857 26677 executor.go:111] Tasks: 0 done / 82 total; 49 can run I0528 04:37:03.211233 26677 executor.go:111] Tasks: 49 done / 82 total; 19 can run I0528 04:37:03.216182 26677 executor.go:111] Tasks: 68 done / 82 total; 8 can run I0528 04:37:03.222867 26677 executor.go:111] Tasks: 76 done / 82 total; 6 can run I0528 04:37:03.223263 26677 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0528 04:37:03.227474 26677 target.go:116] Terraform output is in /tmp/test1654599192/out --- PASS: TestHighAvailabilityGCE (0.26s) === RUN TestComplex I0528 04:37:03.232176 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:03.232304 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:03.232345 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:03.232447 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:03.232490 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:03.232538 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:03.232576 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:03.232607 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:03.232647 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:03.232673 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:03.265490 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:03.269066 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:03.269147 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.707174 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:04.827892 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:04.829978 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:04.830021 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:04.830114 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:04.833725 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:04.835809 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:04.836272 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:04.836299 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:04.836344 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:04.836423 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:04.836478 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:04.839537 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:04.839711 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.839828 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.841416 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.841503 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.841607 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:04.841851 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:04.841980 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0528 04:37:04.847735 26677 target.go:116] Terraform output is in /tmp/test1263875173/out I0528 04:37:04.851284 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:04.851375 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:04.851407 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:04.851454 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:04.851492 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:04.851545 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:04.851597 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:04.851630 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:04.851669 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:04.851696 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:04.887532 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:04.891646 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.891735 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:04.930616 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:05.078361 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:05.078693 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:05.078725 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:05.078793 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:05.080892 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:05.082966 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:05.083445 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:05.083506 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:05.083580 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:05.083680 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:05.083730 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:05.087021 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:05.087191 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.087392 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.088418 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.088855 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.088919 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:05.089778 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:05.090495 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0528 04:37:05.095813 26677 target.go:116] Terraform output is in /tmp/test3026056714/out --- PASS: TestComplex (1.87s) === RUN TestCompress I0528 04:37:05.098415 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:05.098499 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:05.098529 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:05.098560 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:05.098586 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:05.098620 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:05.098672 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:05.098691 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:05.098714 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:05.098732 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:05.113048 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:05.114766 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.114823 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.152339 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:05.236273 26677 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0528 04:37:05.236633 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:05.236650 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:05.236707 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:05.237978 26677 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0528 04:37:05.241028 26677 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0528 04:37:05.241278 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.242123 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.246564 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.247013 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.247085 26677 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0528 04:37:05.247253 26677 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0528 04:37:05.247396 26677 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0528 04:37:05.249952 26677 target.go:116] Terraform output is in /tmp/test4270759779/out --- PASS: TestCompress (0.15s) === RUN TestExternalPolicies I0528 04:37:05.251866 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:05.251940 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:05.251963 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:05.251989 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:05.252011 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:05.252041 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:05.252090 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:05.252107 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:05.252131 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:05.252148 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:05.269602 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:05.271599 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.271656 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.309608 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:05.387052 26677 executor.go:111] Tasks: 0 done / 100 total; 45 can run I0528 04:37:05.390808 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:05.390909 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:05.391074 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:05.395365 26677 executor.go:111] Tasks: 45 done / 100 total; 18 can run I0528 04:37:05.399086 26677 executor.go:111] Tasks: 63 done / 100 total; 26 can run I0528 04:37:05.399737 26677 executor.go:111] Tasks: 89 done / 100 total; 3 can run W0528 04:37:05.399858 26677 api.go:53] PageSize not implemented W0528 04:37:05.399886 26677 api.go:53] PageSize not implemented I0528 04:37:05.403376 26677 executor.go:111] Tasks: 92 done / 100 total; 4 can run I0528 04:37:05.403509 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.403731 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.404644 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.405270 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.405349 26677 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0528 04:37:05.405590 26677 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0528 04:37:05.405728 26677 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0528 04:37:05.409051 26677 target.go:116] Terraform output is in /tmp/test447691191/out --- PASS: TestExternalPolicies (0.16s) === RUN TestMinimalIPv6 I0528 04:37:05.411039 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:05.411117 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:05.411144 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:05.411196 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:05.411226 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:05.411257 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:05.411283 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:05.411301 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:05.411327 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:05.411346 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:05.428637 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:05.430794 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.430853 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.471369 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:05.601821 26677 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0528 04:37:05.602525 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:05.602563 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:05.602716 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:05.604281 26677 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0528 04:37:05.606422 26677 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0528 04:37:05.607145 26677 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0528 04:37:05.607639 26677 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0528 04:37:05.607841 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:05.607905 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:05.608400 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:05.608474 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:05.614093 26677 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0528 04:37:05.614512 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.614712 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.615944 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.616667 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.616751 26677 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0528 04:37:05.616997 26677 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0528 04:37:05.617186 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:05.621837 26677 target.go:116] Terraform output is in /tmp/test2672202041/out --- PASS: TestMinimalIPv6 (0.21s) === RUN TestMinimalIPv6Calico I0528 04:37:05.624937 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:05.625055 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:05.625092 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:05.625168 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:05.625201 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:05.625246 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:05.625285 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:05.625308 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:05.625337 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:05.625359 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:05.652188 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:05.655165 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.655242 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:05.692684 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:06.009529 26677 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0528 04:37:06.010123 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:06.010145 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:06.010222 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:06.015599 26677 executor.go:111] Tasks: 47 done / 126 total; 21 can run I0528 04:37:06.017691 26677 executor.go:111] Tasks: 68 done / 126 total; 33 can run I0528 04:37:06.018162 26677 executor.go:111] Tasks: 101 done / 126 total; 9 can run I0528 04:37:06.018501 26677 executor.go:111] Tasks: 110 done / 126 total; 8 can run I0528 04:37:06.018638 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.018700 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.018752 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.018800 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.021624 26677 executor.go:111] Tasks: 118 done / 126 total; 4 can run I0528 04:37:06.021749 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.021908 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.022778 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.023289 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.023351 26677 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0528 04:37:06.023530 26677 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0528 04:37:06.023619 26677 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0528 04:37:06.027509 26677 target.go:116] Terraform output is in /tmp/test4196183244/out --- PASS: TestMinimalIPv6Calico (0.41s) === RUN TestMinimalIPv6Cilium I0528 04:37:06.031753 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:06.031874 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:06.031914 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:06.031998 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:06.032037 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:06.032084 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:06.032156 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:06.032203 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:06.032235 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:06.032257 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:06.062327 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:06.065739 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.065817 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.105057 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:06.297834 26677 executor.go:111] Tasks: 0 done / 125 total; 47 can run I0528 04:37:06.298466 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:06.298497 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:06.298595 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:06.304593 26677 executor.go:111] Tasks: 47 done / 125 total; 21 can run I0528 04:37:06.307888 26677 executor.go:111] Tasks: 68 done / 125 total; 32 can run I0528 04:37:06.308595 26677 executor.go:111] Tasks: 100 done / 125 total; 9 can run I0528 04:37:06.309149 26677 executor.go:111] Tasks: 109 done / 125 total; 8 can run I0528 04:37:06.309322 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.309443 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.309578 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.309662 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.312702 26677 executor.go:111] Tasks: 117 done / 125 total; 4 can run I0528 04:37:06.312811 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.313025 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.313726 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.315981 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.316104 26677 executor.go:111] Tasks: 121 done / 125 total; 2 can run I0528 04:37:06.316404 26677 executor.go:111] Tasks: 123 done / 125 total; 2 can run I0528 04:37:06.316547 26677 executor.go:111] Tasks: 125 done / 125 total; 0 can run I0528 04:37:06.322556 26677 target.go:116] Terraform output is in /tmp/test256205370/out --- PASS: TestMinimalIPv6Cilium (0.29s) === RUN TestMinimalIPv6NoSubnetPrefix I0528 04:37:06.326604 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:06.326724 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:06.326761 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:06.326846 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:06.326887 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:06.326938 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:06.326979 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:06.327008 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:06.327043 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:06.327070 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:06.357013 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:06.360974 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.361052 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.399186 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:06.543790 26677 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0528 04:37:06.544953 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:06.545002 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:06.545130 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:06.546761 26677 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0528 04:37:06.549128 26677 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0528 04:37:06.549979 26677 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0528 04:37:06.550407 26677 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0528 04:37:06.550646 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.550708 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.550824 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:06.550957 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:06.553820 26677 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0528 04:37:06.553923 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.554085 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.555483 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.555914 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.556028 26677 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0528 04:37:06.556310 26677 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0528 04:37:06.556455 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:06.562356 26677 target.go:116] Terraform output is in /tmp/test1143094300/out --- PASS: TestMinimalIPv6NoSubnetPrefix (0.24s) === RUN TestMinimalWarmPool I0528 04:37:06.566317 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:06.566438 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:06.566475 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:06.566519 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:06.566579 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:06.566633 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:06.566714 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:06.566744 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:06.566780 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:06.566807 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:06.595629 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:06.599117 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.599213 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.638904 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:06.829784 26677 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0528 04:37:06.830516 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:06.830565 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:06.830704 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:06.832178 26677 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0528 04:37:06.836355 26677 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0528 04:37:06.836610 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.836834 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.837703 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.838343 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.838411 26677 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0528 04:37:06.838572 26677 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0528 04:37:06.838725 26677 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0528 04:37:06.846095 26677 target.go:116] Terraform output is in /tmp/test1900143782/out --- PASS: TestMinimalWarmPool (0.28s) === RUN TestMinimalEtcd I0528 04:37:06.848365 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:06.848453 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:06.848483 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:06.848517 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:06.848542 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:06.848575 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:06.848629 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:06.848646 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:06.848669 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:06.848686 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:06.865282 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:06.867102 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.867161 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:06.905217 26677 zones.go:93] ListHostedZonesPages { } W0528 04:37:06.996079 26677 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0528 04:37:06.996139 26677 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 10 W0528 04:37:06.997968 26677 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0528 04:37:07.009544 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:07.010061 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.010100 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.010173 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.012261 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:07.016422 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:07.017258 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.017649 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.021117 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.022630 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.022758 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:07.023051 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:07.023180 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:07.027954 26677 target.go:116] Terraform output is in /tmp/test2614966041/out --- PASS: TestMinimalEtcd (0.18s) === RUN TestMinimalGp3 I0528 04:37:07.031511 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.031651 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.031691 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.031733 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.031772 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.031824 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.031907 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.031935 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.031971 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.032001 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:07.058467 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:07.061656 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.061755 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.101364 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:07.205139 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:07.205917 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.205961 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.206094 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.208068 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:07.211470 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:07.211649 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.211942 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.213234 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.213428 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.213549 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:07.214081 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:07.214401 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:07.218774 26677 target.go:116] Terraform output is in /tmp/test2514006735/out --- PASS: TestMinimalGp3 (0.19s) === RUN TestMinimalLongClusterName I0528 04:37:07.222170 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.222293 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.222330 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.222371 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.222420 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.222468 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.222546 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.222574 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.222610 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.222637 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:07.247539 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:07.250343 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.250425 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:37:07.288755 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:07.382808 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:07.385107 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.385137 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.385226 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.390294 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:07.395451 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:07.395662 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.395943 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.397809 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.398664 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.398780 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:07.399074 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:07.399225 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:07.402089 26677 target.go:116] Terraform output is in /tmp/test2397010845/out --- PASS: TestMinimalLongClusterName (0.18s) === RUN TestExistingSG I0528 04:37:07.404273 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.404359 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.404389 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.404423 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.404448 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.404482 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.404533 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.404550 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.404573 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.404589 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:07.429103 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:07.431157 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.431219 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.431255 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.431290 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.470407 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:07.571107 26677 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0528 04:37:07.571826 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.571862 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.571960 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.574201 26677 executor.go:111] Tasks: 53 done / 146 total; 22 can run I0528 04:37:07.580115 26677 executor.go:111] Tasks: 75 done / 146 total; 50 can run I0528 04:37:07.580986 26677 executor.go:111] Tasks: 125 done / 146 total; 5 can run W0528 04:37:07.581059 26677 api.go:53] PageSize not implemented W0528 04:37:07.581195 26677 api.go:53] PageSize not implemented W0528 04:37:07.581316 26677 api.go:53] PageSize not implemented W0528 04:37:07.581462 26677 api.go:53] PageSize not implemented I0528 04:37:07.589955 26677 executor.go:111] Tasks: 130 done / 146 total; 8 can run I0528 04:37:07.590180 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.590436 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.590696 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.590993 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.592945 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.593040 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.593424 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.593816 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.593926 26677 executor.go:111] Tasks: 138 done / 146 total; 4 can run I0528 04:37:07.594327 26677 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0528 04:37:07.594645 26677 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0528 04:37:07.602264 26677 target.go:116] Terraform output is in /tmp/test871520106/out --- PASS: TestExistingSG (0.20s) === RUN TestBastionAdditionalUserData I0528 04:37:07.606948 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.607081 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.607120 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.607164 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.607208 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.607257 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.607337 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.607376 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.607418 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.607446 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:07.638669 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:07.640632 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.640696 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.640732 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.678494 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:07.770208 26677 executor.go:111] Tasks: 0 done / 121 total; 46 can run I0528 04:37:07.770846 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.770879 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.771000 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.773987 26677 executor.go:111] Tasks: 46 done / 121 total; 26 can run I0528 04:37:07.777429 26677 executor.go:111] Tasks: 72 done / 121 total; 31 can run I0528 04:37:07.778103 26677 executor.go:111] Tasks: 103 done / 121 total; 5 can run W0528 04:37:07.778149 26677 api.go:53] PageSize not implemented W0528 04:37:07.778190 26677 api.go:53] PageSize not implemented W0528 04:37:07.778329 26677 api.go:53] PageSize not implemented I0528 04:37:07.785093 26677 executor.go:111] Tasks: 108 done / 121 total; 7 can run I0528 04:37:07.785259 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.785377 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.785489 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.786221 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.786542 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.786848 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.786911 26677 executor.go:111] Tasks: 115 done / 121 total; 3 can run I0528 04:37:07.787128 26677 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0528 04:37:07.787281 26677 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0528 04:37:07.791007 26677 target.go:116] Terraform output is in /tmp/test1666921991/out --- PASS: TestBastionAdditionalUserData (0.19s) === RUN TestPrivateWeave I0528 04:37:07.793516 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.793599 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.793626 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.793657 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.793683 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.793715 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.793769 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.793786 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.793809 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.793826 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:07.810498 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:07.812400 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.812458 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.812492 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.850021 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:07.969741 26677 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0528 04:37:07.970735 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:07.970767 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:07.970888 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:07.972676 26677 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0528 04:37:07.976124 26677 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0528 04:37:07.976610 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.976824 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.976929 26677 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0528 04:37:07.976977 26677 api.go:53] PageSize not implemented W0528 04:37:07.977009 26677 api.go:53] PageSize not implemented I0528 04:37:07.981685 26677 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0528 04:37:07.981943 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.982131 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.983875 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.983980 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:07.984087 26677 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0528 04:37:07.984342 26677 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0528 04:37:07.984482 26677 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0528 04:37:07.988198 26677 target.go:116] Terraform output is in /tmp/test3582195845/out --- PASS: TestPrivateWeave (0.20s) === RUN TestPrivateFlannel I0528 04:37:07.991014 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:07.991095 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:07.991123 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:07.991180 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:07.991205 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:07.991239 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:07.991265 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:07.991281 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:07.991307 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:07.991326 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:08.012458 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:08.015813 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.015909 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.015968 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.054858 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:08.229603 26677 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0528 04:37:08.230513 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:08.230565 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:08.230700 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:08.233764 26677 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0528 04:37:08.237059 26677 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0528 04:37:08.237430 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.237578 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.237802 26677 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0528 04:37:08.237881 26677 api.go:53] PageSize not implemented W0528 04:37:08.237949 26677 api.go:53] PageSize not implemented I0528 04:37:08.242690 26677 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0528 04:37:08.242866 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.243088 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.244718 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.245456 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.245562 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:08.245871 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:08.246063 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:08.256592 26677 target.go:116] Terraform output is in /tmp/test614465302/out --- PASS: TestPrivateFlannel (0.27s) === RUN TestPrivateCalico I0528 04:37:08.261465 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:08.261621 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:08.261663 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:08.261752 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:08.261797 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:08.261844 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:08.261889 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:08.261920 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:08.261956 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:08.261984 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:08.291640 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:08.294948 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.295056 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.295132 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.334540 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:08.730986 26677 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0528 04:37:08.731322 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:08.731365 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:08.731459 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:08.734667 26677 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0528 04:37:08.737292 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0528 04:37:08.737828 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.737965 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.738040 26677 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0528 04:37:08.738228 26677 api.go:53] PageSize not implemented W0528 04:37:08.738342 26677 api.go:53] PageSize not implemented I0528 04:37:08.743285 26677 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0528 04:37:08.743587 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.743849 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.745197 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.745714 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.745789 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0528 04:37:08.746137 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:08.746298 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:08.750315 26677 target.go:116] Terraform output is in /tmp/test4139607494/out --- PASS: TestPrivateCalico (0.49s) === RUN TestPrivateCilium I0528 04:37:08.753325 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:08.753418 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:08.753445 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:08.753481 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:08.753508 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:08.753546 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:08.753599 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:08.753615 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:08.753638 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:08.753655 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:08.781077 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:08.784703 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.784797 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.784872 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.824848 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:08.924426 26677 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0528 04:37:08.928915 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:08.928949 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:08.929150 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:08.931540 26677 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0528 04:37:08.933785 26677 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0528 04:37:08.934008 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.934219 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.934329 26677 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0528 04:37:08.934402 26677 api.go:53] PageSize not implemented W0528 04:37:08.934430 26677 api.go:53] PageSize not implemented I0528 04:37:08.939673 26677 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0528 04:37:08.939923 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.940151 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.941498 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.941729 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.941816 26677 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0528 04:37:08.942126 26677 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0528 04:37:08.942269 26677 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0528 04:37:08.945919 26677 target.go:116] Terraform output is in /tmp/test1490701895/out --- PASS: TestPrivateCilium (0.19s) === RUN TestPrivateCilium2 I0528 04:37:08.948341 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:08.948442 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:08.948480 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:08.948557 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:08.948595 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:08.948645 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:08.948683 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:08.948712 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:08.948752 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:08.948777 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:08.973176 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:08.976780 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.976872 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:08.976932 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.016179 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:09.616079 26677 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0528 04:37:09.616697 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:09.616743 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:09.616855 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:09.619757 26677 executor.go:111] Tasks: 51 done / 124 total; 26 can run I0528 04:37:09.624303 26677 executor.go:111] Tasks: 77 done / 124 total; 32 can run I0528 04:37:09.624620 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.624740 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.624853 26677 executor.go:111] Tasks: 109 done / 124 total; 5 can run W0528 04:37:09.625049 26677 api.go:53] PageSize not implemented W0528 04:37:09.625154 26677 api.go:53] PageSize not implemented I0528 04:37:09.627785 26677 executor.go:111] Tasks: 114 done / 124 total; 6 can run I0528 04:37:09.627897 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.628028 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.628860 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.629749 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.629825 26677 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0528 04:37:09.630019 26677 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0528 04:37:09.630143 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:09.634226 26677 target.go:116] Terraform output is in /tmp/test615003714/out --- PASS: TestPrivateCilium2 (0.69s) === RUN TestPrivateCiliumAdvanced I0528 04:37:09.639811 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:09.639911 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:09.639942 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:09.639976 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:09.640007 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:09.640040 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:09.640097 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:09.640113 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:09.640140 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:09.640158 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:09.661216 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:09.663532 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.663591 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.663626 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.701391 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:09.822823 26677 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0528 04:37:09.823145 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:09.823191 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:09.823300 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:09.830148 26677 executor.go:111] Tasks: 53 done / 126 total; 26 can run I0528 04:37:09.833858 26677 executor.go:111] Tasks: 79 done / 126 total; 32 can run I0528 04:37:09.834272 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.834507 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.834653 26677 executor.go:111] Tasks: 111 done / 126 total; 5 can run W0528 04:37:09.834715 26677 api.go:53] PageSize not implemented W0528 04:37:09.834875 26677 api.go:53] PageSize not implemented I0528 04:37:09.840804 26677 executor.go:111] Tasks: 116 done / 126 total; 6 can run I0528 04:37:09.841084 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.841352 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.843023 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.843775 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.843886 26677 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0528 04:37:09.844191 26677 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0528 04:37:09.844378 26677 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0528 04:37:09.850594 26677 target.go:116] Terraform output is in /tmp/test3629114838/out --- PASS: TestPrivateCiliumAdvanced (0.21s) === RUN TestPrivateCiliumENI I0528 04:37:09.854676 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:09.854804 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:09.854845 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:09.854889 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:09.854945 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:09.854998 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:09.855080 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:09.855110 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:09.855150 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:09.855188 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:09.884160 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:09.887224 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.887297 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.887352 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:09.926171 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:10.074376 26677 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0528 04:37:10.074643 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:10.074672 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:10.074753 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:10.081161 26677 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0528 04:37:10.084835 26677 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0528 04:37:10.085273 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.085503 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.085612 26677 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0528 04:37:10.085654 26677 api.go:53] PageSize not implemented W0528 04:37:10.085681 26677 api.go:53] PageSize not implemented I0528 04:37:10.090677 26677 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0528 04:37:10.090971 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.091187 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.092511 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.092772 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.092875 26677 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0528 04:37:10.093154 26677 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0528 04:37:10.093324 26677 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0528 04:37:10.096910 26677 target.go:116] Terraform output is in /tmp/test1683812522/out --- PASS: TestPrivateCiliumENI (0.24s) === RUN TestPrivateCanal I0528 04:37:10.099132 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:10.099209 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:10.099236 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:10.099292 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:10.099318 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:10.099349 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:10.099374 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:10.099390 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:10.099414 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:10.099432 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:10.121999 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:10.125190 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.125263 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.125312 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.166085 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:10.578803 26677 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0528 04:37:10.580297 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:10.580338 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:10.580465 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:10.586528 26677 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0528 04:37:10.589910 26677 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0528 04:37:10.590453 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.590614 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.590722 26677 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0528 04:37:10.590777 26677 api.go:53] PageSize not implemented W0528 04:37:10.590812 26677 api.go:53] PageSize not implemented I0528 04:37:10.595846 26677 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0528 04:37:10.596007 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.596188 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.597501 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.598940 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.599052 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:10.599346 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:10.599539 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:10.605995 26677 target.go:116] Terraform output is in /tmp/test1774859604/out --- PASS: TestPrivateCanal (0.51s) === RUN TestPrivateKopeio I0528 04:37:10.611545 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:10.611704 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:10.611744 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:10.611788 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:10.611829 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:10.611875 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:10.611957 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:10.611986 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:10.612024 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:10.612051 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:10.645890 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:10.649869 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.649956 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.650016 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.689060 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:10.804904 26677 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0528 04:37:10.805487 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:10.805521 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:10.805613 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:10.812529 26677 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0528 04:37:10.816673 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0528 04:37:10.817132 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.817383 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.817529 26677 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0528 04:37:10.817680 26677 api.go:53] PageSize not implemented W0528 04:37:10.817764 26677 api.go:53] PageSize not implemented I0528 04:37:10.822883 26677 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0528 04:37:10.823051 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.823226 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.825234 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.825731 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.825840 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0528 04:37:10.826144 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:10.826328 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:10.832664 26677 target.go:116] Terraform output is in /tmp/test930059921/out --- PASS: TestPrivateKopeio (0.22s) === RUN TestUnmanaged I0528 04:37:10.836556 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:10.836680 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:10.836718 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:10.836762 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:10.836802 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:10.836849 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:10.836932 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:10.836961 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:10.836999 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:10.837026 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:10.867671 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:10.870583 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.870660 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.870715 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:10.910553 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:11.021193 26677 executor.go:111] Tasks: 0 done / 110 total; 45 can run I0528 04:37:11.021772 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:11.021798 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:11.021888 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:11.023296 26677 executor.go:111] Tasks: 45 done / 110 total; 23 can run I0528 04:37:11.026685 26677 executor.go:111] Tasks: 68 done / 110 total; 29 can run I0528 04:37:11.027298 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.027516 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.027627 26677 executor.go:111] Tasks: 97 done / 110 total; 4 can run W0528 04:37:11.027667 26677 api.go:53] PageSize not implemented W0528 04:37:11.027806 26677 api.go:53] PageSize not implemented I0528 04:37:11.030621 26677 executor.go:111] Tasks: 101 done / 110 total; 5 can run I0528 04:37:11.030733 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.030855 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.032624 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.033456 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.033573 26677 executor.go:111] Tasks: 106 done / 110 total; 2 can run I0528 04:37:11.033951 26677 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0528 04:37:11.034157 26677 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0528 04:37:11.041598 26677 target.go:116] Terraform output is in /tmp/test47434966/out --- PASS: TestUnmanaged (0.21s) === RUN TestPrivateSharedSubnet I0528 04:37:11.044093 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:11.044179 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:11.044208 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:11.044239 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:11.044265 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:11.044298 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:11.044350 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:11.044366 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:11.044389 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:11.044407 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:11.061175 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:11.063053 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.063105 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.063140 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.100749 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:11.172840 26677 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0528 04:37:11.173709 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:11.173750 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:11.173872 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:11.176437 26677 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0528 04:37:11.176843 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:11.183830 26677 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0528 04:37:11.184453 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.184640 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.184779 26677 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0528 04:37:11.184950 26677 api.go:53] PageSize not implemented W0528 04:37:11.185095 26677 api.go:53] PageSize not implemented I0528 04:37:11.189775 26677 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0528 04:37:11.189963 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.190251 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.192716 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.193880 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.194005 26677 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0528 04:37:11.194398 26677 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0528 04:37:11.194614 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:11.200642 26677 target.go:116] Terraform output is in /tmp/test3619245513/out --- PASS: TestPrivateSharedSubnet (0.16s) === RUN TestPrivateSharedIP I0528 04:37:11.204351 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:11.204479 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:11.204517 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:11.204558 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:11.204596 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:11.204644 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:11.204725 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:11.204766 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:11.204805 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:11.204832 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:11.233768 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:11.236853 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.236931 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.236982 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.275732 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:11.377344 26677 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0528 04:37:11.380525 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:11.380574 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:11.380662 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:11.384056 26677 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0528 04:37:11.384464 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:11.387634 26677 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0528 04:37:11.388156 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.388360 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.388459 26677 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0528 04:37:11.388501 26677 api.go:53] PageSize not implemented W0528 04:37:11.388662 26677 api.go:53] PageSize not implemented I0528 04:37:11.393493 26677 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0528 04:37:11.393767 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.393999 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.395398 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.396744 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.396852 26677 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0528 04:37:11.397176 26677 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0528 04:37:11.397427 26677 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0528 04:37:11.403438 26677 target.go:116] Terraform output is in /tmp/test3405535291/out --- PASS: TestPrivateSharedIP (0.20s) === RUN TestPrivateDns1 I0528 04:37:11.407191 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:11.407312 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:11.407350 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:11.407392 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:11.407433 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:11.407483 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:11.407564 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:11.407592 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:11.407628 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:11.407656 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:11.440178 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.440258 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.440315 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.549907 26677 executor.go:111] Tasks: 0 done / 120 total; 46 can run I0528 04:37:11.553226 26677 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0528 04:37:11.553461 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "internal.example.com" I0528 04:37:11.553514 26677 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0528 04:37:11.553647 26677 dnszone.go:248] Existing zone "internal.example.com." found; will configure TF to reuse I0528 04:37:11.553676 26677 dnszone.go:271] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0528 04:37:11.553947 26677 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0528 04:37:11.554347 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.554568 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.557721 26677 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0528 04:37:11.557855 26677 api.go:53] PageSize not implemented W0528 04:37:11.557958 26677 api.go:53] PageSize not implemented I0528 04:37:11.566334 26677 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0528 04:37:11.566517 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.566750 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.567931 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.569199 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.569308 26677 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0528 04:37:11.569605 26677 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0528 04:37:11.569788 26677 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0528 04:37:11.577270 26677 target.go:116] Terraform output is in /tmp/test1348564182/out --- PASS: TestPrivateDns1 (0.17s) === RUN TestPrivateDns2 I0528 04:37:11.581253 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:11.581373 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:11.581413 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:11.581455 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:11.581496 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:11.581548 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:11.581650 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:11.581683 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:11.581742 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:11.581775 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:11.613899 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.613978 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.614033 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.752426 26677 executor.go:111] Tasks: 0 done / 116 total; 44 can run I0528 04:37:11.759693 26677 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0528 04:37:11.760018 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:11.760165 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "private.example.com" I0528 04:37:11.760198 26677 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0528 04:37:11.760304 26677 dnszone.go:248] Existing zone "private.example.com." found; will configure TF to reuse I0528 04:37:11.760337 26677 dnszone.go:262] VPC "vpc-12345678" already associated with zone "private.example.com." I0528 04:37:11.760421 26677 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0528 04:37:11.760831 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.760989 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.763737 26677 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0528 04:37:11.763795 26677 api.go:53] PageSize not implemented W0528 04:37:11.763924 26677 api.go:53] PageSize not implemented I0528 04:37:11.768706 26677 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0528 04:37:11.768941 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.769139 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.770152 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.772473 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.772632 26677 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0528 04:37:11.773122 26677 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0528 04:37:11.773339 26677 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0528 04:37:11.779425 26677 target.go:116] Terraform output is in /tmp/test4067710676/out --- PASS: TestPrivateDns2 (0.20s) === RUN TestDiscoveryFeatureGate I0528 04:37:11.783476 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:11.783612 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:11.783665 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:11.783770 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:11.783818 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:11.783871 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:11.783917 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:11.783951 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:11.783997 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:11.784030 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:11.814522 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:11.818041 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.818141 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:11.858384 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:12.025741 26677 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0528 04:37:12.026152 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:12.026185 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:12.026273 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:12.027974 26677 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0528 04:37:12.030885 26677 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0528 04:37:12.031152 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.031340 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.032495 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.032953 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.033031 26677 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0528 04:37:12.033219 26677 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0528 04:37:12.033380 26677 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0528 04:37:12.036649 26677 target.go:116] Terraform output is in /tmp/test4269218070/out --- PASS: TestDiscoveryFeatureGate (0.26s) === RUN TestVFSServiceAccountIssuerDiscovery I0528 04:37:12.039416 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:12.039502 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:12.039531 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:12.039560 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:12.039583 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:12.039613 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:12.039667 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:12.039685 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:12.039708 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:12.039725 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:12.055764 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:12.057653 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.057726 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.096617 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:12.202316 26677 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0528 04:37:12.202838 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:12.202866 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:12.202952 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:12.204556 26677 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0528 04:37:12.207515 26677 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0528 04:37:12.207813 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.208036 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.209290 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.209420 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.209505 26677 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0528 04:37:12.209732 26677 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0528 04:37:12.209875 26677 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0528 04:37:12.212689 26677 target.go:116] Terraform output is in /tmp/test3230912698/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.18s) === RUN TestAWSLBController I0528 04:37:12.214804 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:12.214892 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:12.214934 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:12.214970 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:12.214994 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:12.215023 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:12.215079 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:12.215097 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:12.215120 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:12.215138 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:12.234799 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:12.236659 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.236726 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.275794 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:12.824937 26677 executor.go:111] Tasks: 0 done / 96 total; 49 can run I0528 04:37:12.825483 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:12.825585 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:12.825727 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:12.830591 26677 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0528 04:37:12.834552 26677 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0528 04:37:12.834910 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.835091 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.835821 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.836718 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.836784 26677 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0528 04:37:12.836965 26677 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0528 04:37:12.837076 26677 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0528 04:37:12.840563 26677 target.go:116] Terraform output is in /tmp/test541765201/out --- PASS: TestAWSLBController (0.63s) === RUN TestManyAddons I0528 04:37:12.845530 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:12.845628 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:12.845658 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:12.845688 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:12.845713 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:12.845745 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:12.845798 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:12.845815 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:12.845844 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:12.845862 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:12.862725 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:12.864885 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.864976 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:12.902826 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:13.539303 26677 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0528 04:37:13.540202 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:13.540242 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:13.540334 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:13.542977 26677 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0528 04:37:13.548942 26677 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0528 04:37:13.549158 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.549433 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.550136 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.551008 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.551094 26677 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0528 04:37:13.551287 26677 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0528 04:37:13.551522 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:13.557530 26677 target.go:116] Terraform output is in /tmp/test2692921468/out --- PASS: TestManyAddons (0.72s) === RUN TestManyAddonsCCMIRSA I0528 04:37:13.566267 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:13.566395 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:13.566435 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:13.566534 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:13.566575 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:13.566624 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:13.566664 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:13.566695 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:13.566731 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:13.566767 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:13.597613 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:13.601498 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.601602 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:13.640754 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:14.293756 26677 executor.go:111] Tasks: 0 done / 123 total; 61 can run I0528 04:37:14.294545 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:14.294582 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:14.294685 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:14.297465 26677 executor.go:111] Tasks: 61 done / 123 total; 30 can run I0528 04:37:14.300954 26677 executor.go:111] Tasks: 91 done / 123 total; 26 can run I0528 04:37:14.301198 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.301307 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.302109 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.302889 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.302966 26677 executor.go:111] Tasks: 117 done / 123 total; 2 can run I0528 04:37:14.303154 26677 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0528 04:37:14.303335 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:14.313621 26677 target.go:116] Terraform output is in /tmp/test180126542/out --- PASS: TestManyAddonsCCMIRSA (0.76s) === RUN TestManyAddonsCCMIRSA23 I0528 04:37:14.323091 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:14.323223 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:14.323265 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:14.323367 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:14.323413 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:14.323465 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:14.323510 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:14.323544 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:14.323584 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:14.323613 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:14.350762 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0528 04:37:14.354240 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.354328 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:14.396553 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:15.051254 26677 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0528 04:37:15.051990 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:15.052032 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:15.052123 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:15.053921 26677 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0528 04:37:15.057931 26677 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0528 04:37:15.058353 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.059281 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.061539 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.063555 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.063637 26677 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0528 04:37:15.063845 26677 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0528 04:37:15.064006 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:15.071074 26677 target.go:116] Terraform output is in /tmp/test430701385/out --- PASS: TestManyAddonsCCMIRSA23 (0.76s) === RUN TestManyAddonsCCMIRSA24 I0528 04:37:15.079710 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:15.079832 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:15.079872 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:15.079961 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:15.080000 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:15.080049 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:15.080092 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:15.080121 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:15.080157 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:15.080185 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:15.106146 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:15.109748 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.109830 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.148155 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:15.810405 26677 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0528 04:37:15.814627 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:15.814656 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:15.814723 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:15.817547 26677 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0528 04:37:15.820440 26677 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0528 04:37:15.820659 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.820786 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.821625 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.822283 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.822345 26677 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0528 04:37:15.822523 26677 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0528 04:37:15.822720 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:15.831979 26677 target.go:116] Terraform output is in /tmp/test2238725130/out --- PASS: TestManyAddonsCCMIRSA24 (0.76s) === RUN TestManyAddonsCCMIRSA25 I0528 04:37:15.838295 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:15.838383 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:15.838413 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:15.838478 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:15.838505 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:15.838538 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:15.838567 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:15.838584 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:15.838607 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:15.838624 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:15.854768 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:37:15.857037 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.857094 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:15.896509 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:16.534609 26677 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0528 04:37:16.537291 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:16.537339 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:16.537424 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:16.541142 26677 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0528 04:37:16.546367 26677 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0528 04:37:16.546556 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.546763 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.547573 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.549028 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.549116 26677 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0528 04:37:16.549320 26677 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0528 04:37:16.549496 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:16.557813 26677 target.go:116] Terraform output is in /tmp/test3513649527/out --- PASS: TestManyAddonsCCMIRSA25 (0.73s) === RUN TestManyAddonsCCMIRSA26 I0528 04:37:16.563731 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:16.563876 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:16.563929 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:16.563980 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:16.564027 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:16.564083 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:16.564177 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:16.564207 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:16.564250 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:16.564282 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:16.580745 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:37:16.582977 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.583031 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:16.621340 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:17.348371 26677 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0528 04:37:17.355912 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:17.355946 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:17.356005 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:17.357702 26677 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0528 04:37:17.364351 26677 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0528 04:37:17.364572 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:17.364781 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:17.366198 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:17.367487 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:17.367601 26677 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0528 04:37:17.367892 26677 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0528 04:37:17.368216 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:17.380418 26677 target.go:116] Terraform output is in /tmp/test1165437522/out --- PASS: TestManyAddonsCCMIRSA26 (0.83s) === RUN TestManyAddonsGCE I0528 04:37:17.390393 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:17.983677 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:17.983745 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:17.983822 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/main" W0528 04:37:17.983842 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/events" W0528 04:37:17.983856 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com" W0528 04:37:17.983904 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:17.984854 26677 executor.go:111] Tasks: 0 done / 67 total; 44 can run I0528 04:37:17.991601 26677 executor.go:111] Tasks: 44 done / 67 total; 17 can run I0528 04:37:17.994568 26677 executor.go:111] Tasks: 61 done / 67 total; 4 can run I0528 04:37:17.996222 26677 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0528 04:37:17.996333 26677 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0528 04:37:17.998668 26677 target.go:116] Terraform output is in /tmp/test3102847920/out --- PASS: TestManyAddonsGCE (0.61s) === RUN TestCCM I0528 04:37:18.002699 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:18.002779 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:18.002812 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:18.002871 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:18.002897 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:18.002925 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:18.002953 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:18.002971 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:18.002994 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:18.003010 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:18.018704 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:18.020816 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.020871 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.058417 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:18.704487 26677 executor.go:111] Tasks: 0 done / 110 total; 53 can run I0528 04:37:18.704728 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:18.704749 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:18.704821 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:18.710393 26677 executor.go:111] Tasks: 53 done / 110 total; 24 can run I0528 04:37:18.713991 26677 executor.go:111] Tasks: 77 done / 110 total; 27 can run I0528 04:37:18.714228 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.714347 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.715177 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.717226 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.717336 26677 executor.go:111] Tasks: 104 done / 110 total; 2 can run I0528 04:37:18.717578 26677 executor.go:111] Tasks: 106 done / 110 total; 4 can run I0528 04:37:18.717791 26677 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0528 04:37:18.722181 26677 target.go:116] Terraform output is in /tmp/test1780183442/out --- PASS: TestCCM (0.73s) === RUN TestExternalDNS I0528 04:37:18.728308 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:18.728412 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:18.728441 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:18.728476 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:18.728501 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:18.728532 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:18.728585 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:18.728604 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:18.728629 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:18.728646 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:18.743422 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:18.745174 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.745233 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.782739 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:18.860711 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:18.861182 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:18.861217 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:18.861314 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:18.862995 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:18.866018 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:18.866144 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.866286 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.867150 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.867632 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.867703 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:18.867885 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:18.868009 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:18.870856 26677 target.go:116] Terraform output is in /tmp/test3885985177/out --- PASS: TestExternalDNS (0.14s) === RUN TestExternalDNSIRSA I0528 04:37:18.872993 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:18.873072 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:18.873096 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:18.873123 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:18.873147 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:18.873178 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:18.873231 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:18.873250 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:18.873274 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:18.873292 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:18.889383 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:18.891251 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.891309 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:18.928549 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:19.009515 26677 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0528 04:37:19.010162 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:19.010196 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:19.010285 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:19.011659 26677 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0528 04:37:19.014597 26677 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0528 04:37:19.014756 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.014858 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.015584 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.016267 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.016338 26677 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0528 04:37:19.016518 26677 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0528 04:37:19.016635 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0528 04:37:19.019318 26677 target.go:116] Terraform output is in /tmp/test3012507492/out --- PASS: TestExternalDNSIRSA (0.15s) === RUN TestKarpenter I0528 04:37:19.021510 26677 featureflag.go:166] FeatureFlag "Karpenter"=true I0528 04:37:19.021537 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:19.021623 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:19.021649 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:19.021676 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:19.021701 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:19.021734 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:19.021781 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:19.021798 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:19.021823 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:19.021841 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:19.043779 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:19.045888 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.045944 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.045978 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.083762 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:19.212701 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.212796 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.212864 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.286167 26677 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0528 04:37:19.287231 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:19.287286 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:19.288538 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:19.294398 26677 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0528 04:37:19.298061 26677 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0528 04:37:19.298320 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.298520 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.298708 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.299053 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.300060 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.300152 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.300816 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.302278 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.302414 26677 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0528 04:37:19.302748 26677 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0528 04:37:19.302957 26677 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0528 04:37:19.309835 26677 target.go:116] Terraform output is in /tmp/test2241965225/out I0528 04:37:19.315446 26677 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.29s) === RUN TestSharedSubnet I0528 04:37:19.315561 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:19.315687 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:19.315728 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:19.315772 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:19.315821 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:19.315872 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:19.315957 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:19.315987 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:19.316032 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:19.316062 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:19.341819 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:19.344630 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.344708 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.383975 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:19.492208 26677 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0528 04:37:19.492484 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:19.492522 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:19.492670 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:19.495836 26677 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0528 04:37:19.496344 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.500702 26677 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0528 04:37:19.501147 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.501424 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.503030 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.503652 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.503759 26677 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0528 04:37:19.504053 26677 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0528 04:37:19.504235 26677 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0528 04:37:19.508573 26677 target.go:116] Terraform output is in /tmp/test4249293989/out --- PASS: TestSharedSubnet (0.20s) === RUN TestSharedVPC I0528 04:37:19.512251 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:19.512380 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:19.512421 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:19.512465 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:19.512507 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:19.512558 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:19.512640 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:19.512672 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:19.512712 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:19.512739 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:19.542602 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:19.546145 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.546234 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.584672 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:19.691896 26677 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0528 04:37:19.692499 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:19.692530 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:19.692602 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:19.695325 26677 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0528 04:37:19.695800 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.700503 26677 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0528 04:37:19.700887 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.701146 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.701947 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.703980 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.704093 26677 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0528 04:37:19.704382 26677 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0528 04:37:19.704577 26677 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0528 04:37:19.712449 26677 target.go:116] Terraform output is in /tmp/test1093013075/out --- PASS: TestSharedVPC (0.20s) === RUN TestSharedVPCIPv6 I0528 04:37:19.714489 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:19.714566 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:19.714593 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:19.714651 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:19.714679 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:19.714708 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:19.714732 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:19.714750 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:19.714773 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:19.714790 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:19.716064 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:19.716099 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.732681 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:19.732713 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.732771 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:19.735479 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.735534 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.773824 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:19.906763 26677 executor.go:111] Tasks: 0 done / 121 total; 45 can run I0528 04:37:19.907728 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:19.907777 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:19.907941 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:19.910438 26677 executor.go:111] Tasks: 45 done / 121 total; 25 can run I0528 04:37:19.910860 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.911039 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:19.913811 26677 executor.go:111] Tasks: 70 done / 121 total; 33 can run I0528 04:37:19.914585 26677 executor.go:111] Tasks: 103 done / 121 total; 6 can run I0528 04:37:19.914737 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:19.914824 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:19.914954 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:19.915038 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:19.919619 26677 executor.go:111] Tasks: 109 done / 121 total; 8 can run I0528 04:37:19.919907 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.920117 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.921403 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.922810 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.922918 26677 executor.go:111] Tasks: 117 done / 121 total; 2 can run I0528 04:37:19.923221 26677 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0528 04:37:19.923410 26677 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0528 04:37:19.934064 26677 target.go:116] Terraform output is in /tmp/test211800265/out --- PASS: TestSharedVPCIPv6 (0.22s) === RUN TestExistingIAM I0528 04:37:19.938633 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:19.938765 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:19.938848 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:19.938919 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:19.938964 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:19.939013 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:19.939096 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:19.939127 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:19.939165 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:19.939192 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:19.971538 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:19.973898 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.973989 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.974029 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:19.974063 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.014509 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:20.105279 26677 executor.go:111] Tasks: 0 done / 101 total; 51 can run I0528 04:37:20.105819 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:20.105852 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:20.105976 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:20.107874 26677 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0528 04:37:20.117589 26677 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0528 04:37:20.117938 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.118154 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.118467 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.118717 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.119710 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.119778 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.120140 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.121044 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.121161 26677 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0528 04:37:20.121490 26677 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0528 04:37:20.121681 26677 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0528 04:37:20.128038 26677 target.go:116] Terraform output is in /tmp/test2807102873/out --- PASS: TestExistingIAM (0.19s) === RUN TestPhaseNetwork I0528 04:37:20.132130 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.132260 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.132305 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.132352 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.132396 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.132451 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.132495 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.132527 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.132567 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.132597 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.162287 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:20.165262 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.165344 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.165401 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.204062 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:20.292897 26677 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0528 04:37:20.293276 26677 executor.go:111] Tasks: 46 done / 119 total; 26 can run I0528 04:37:20.293654 26677 executor.go:111] Tasks: 72 done / 119 total; 32 can run I0528 04:37:20.293942 26677 executor.go:111] Tasks: 104 done / 119 total; 5 can run I0528 04:37:20.294103 26677 executor.go:111] Tasks: 109 done / 119 total; 6 can run I0528 04:37:20.294231 26677 executor.go:111] Tasks: 115 done / 119 total; 2 can run I0528 04:37:20.294309 26677 executor.go:111] Tasks: 117 done / 119 total; 2 can run I0528 04:37:20.294354 26677 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0528 04:37:20.295087 26677 target.go:116] Terraform output is in /tmp/test194056756/out --- PASS: TestPhaseNetwork (0.16s) === RUN TestExternalLoadBalancer I0528 04:37:20.295471 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.295554 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.295582 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.295613 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.295640 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.295673 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.295763 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.295787 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.295810 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.295829 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.322237 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:20.324221 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.324301 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.366705 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:20.471176 26677 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0528 04:37:20.472292 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:20.472328 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:20.472427 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:20.474749 26677 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0528 04:37:20.479361 26677 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0528 04:37:20.479631 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.479834 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.480800 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.481371 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.481436 26677 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0528 04:37:20.481651 26677 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0528 04:37:20.481811 26677 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0528 04:37:20.485258 26677 target.go:116] Terraform output is in /tmp/test3071963429/out --- PASS: TestExternalLoadBalancer (0.20s) === RUN TestPhaseIAM integration_test.go:920: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:928: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0528 04:37:20.491118 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.491279 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.491325 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.491389 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.491450 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.491519 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.491650 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.491698 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.491753 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.491786 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.519937 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:20.522062 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522122 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522154 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522185 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522214 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522240 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.522272 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.559934 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:20.642582 26677 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0528 04:37:20.643122 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:20.643161 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:20.643254 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:20.645173 26677 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0528 04:37:20.653968 26677 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0528 04:37:20.654179 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.654370 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.654497 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.654697 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.655675 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.656101 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.656621 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.656793 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.656893 26677 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0528 04:37:20.657181 26677 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0528 04:37:20.657362 26677 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0528 04:37:20.664180 26677 target.go:116] Terraform output is in /tmp/test3857192792/out --- PASS: TestMixedInstancesASG (0.18s) === RUN TestMixedInstancesSpotASG I0528 04:37:20.668510 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.668638 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.668679 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.668731 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.668775 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.668826 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.668918 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.668958 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.669002 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.669033 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.699358 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:20.701426 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701484 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701518 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701549 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701589 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701618 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.701645 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.740235 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:20.831559 26677 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0528 04:37:20.832212 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:20.832245 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:20.832366 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:20.834026 26677 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0528 04:37:20.838455 26677 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0528 04:37:20.838836 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.839026 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.839260 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.839445 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.840314 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.840404 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.840763 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.841612 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.841719 26677 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0528 04:37:20.842006 26677 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0528 04:37:20.842194 26677 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0528 04:37:20.846120 26677 target.go:116] Terraform output is in /tmp/test3886393585/out --- PASS: TestMixedInstancesSpotASG (0.18s) === RUN TestAdditionalObjects I0528 04:37:20.849906 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.850073 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.850116 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.850184 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.850232 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.850294 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.850387 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.850433 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.850481 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.850510 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.877893 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0528 04:37:20.879870 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.879936 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:20.917662 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:21.062942 26677 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0528 04:37:21.063652 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:21.063678 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:21.063769 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:21.065294 26677 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0528 04:37:21.069927 26677 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0528 04:37:21.070234 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.070442 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.071274 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.072215 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.072292 26677 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0528 04:37:21.072489 26677 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0528 04:37:21.072623 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0528 04:37:21.075557 26677 target.go:116] Terraform output is in /tmp/test2857025967/out --- PASS: TestAdditionalObjects (0.23s) === RUN TestContainerd I0528 04:37:21.077778 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:21.077865 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:21.077893 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:21.077924 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:21.077952 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:21.077988 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:21.078042 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:21.078060 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:21.078085 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:21.078103 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:21.093867 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:21.096710 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.096796 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.134847 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:21.230450 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:21.233883 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:21.233919 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:21.234025 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:21.238282 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:21.243559 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:21.243895 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.244098 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.245615 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.246136 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.246222 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:21.246473 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:21.246604 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:21.251103 26677 target.go:116] Terraform output is in /tmp/test332716297/out --- PASS: TestContainerd (0.18s) === RUN TestContainerdCustom I0528 04:37:21.254599 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:21.254738 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:21.254784 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:21.254840 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:21.254882 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:21.254945 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:21.255036 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:21.255072 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:21.255113 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:21.255142 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:21.284644 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:21.287971 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.288067 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.326683 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:21.413647 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:21.414028 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:21.414065 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:21.414140 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:21.415920 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:21.420325 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:21.420485 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.420766 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.421467 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.422332 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.422411 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:21.422620 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:21.422796 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:21.425395 26677 target.go:116] Terraform output is in /tmp/test833773806/out --- PASS: TestContainerdCustom (0.17s) === RUN TestDockerCustom I0528 04:37:21.427430 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:21.427518 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:21.427546 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:21.427578 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:21.427602 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:21.427635 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:21.427688 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:21.427707 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:21.427730 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:21.427747 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:21.447049 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:21.449018 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.449080 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.488200 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:21.585892 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:21.586262 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:21.586297 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:21.586364 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:21.588323 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:21.592661 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:21.592807 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.593027 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.593913 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.594880 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.594987 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:21.595197 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:21.595318 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:21.598261 26677 target.go:116] Terraform output is in /tmp/test1166377113/out --- PASS: TestDockerCustom (0.17s) === RUN TestAPIServerNodes I0528 04:37:21.600494 26677 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0528 04:37:21.600509 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:21.600587 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:21.600614 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:21.600643 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:21.600666 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:21.600697 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:21.600759 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:21.600777 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:21.600799 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:21.600817 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:21.621754 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.17 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.17 ********************************************************************************* I0528 04:37:21.623946 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.624007 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:21.624040 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.844349 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:22.959468 26677 executor.go:111] Tasks: 0 done / 100 total; 47 can run I0528 04:37:22.959664 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:22.959681 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:22.959765 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:22.965641 26677 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0528 04:37:22.968927 26677 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0528 04:37:22.969265 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.969386 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.969619 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.970253 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.970854 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.972452 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:22.972547 26677 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0528 04:37:22.972860 26677 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0528 04:37:22.973024 26677 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0528 04:37:22.976119 26677 target.go:116] Terraform output is in /tmp/test1676759210/out I0528 04:37:22.978454 26677 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.38s) === RUN TestNTHIMDSProcessor I0528 04:37:22.978535 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:22.978619 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:22.978645 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:22.978674 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:22.978701 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:22.978733 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:22.978784 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:22.978800 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:22.978823 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:22.978839 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:23.000219 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:23.003309 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.003399 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.041391 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:23.161678 26677 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0528 04:37:23.162399 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:23.162419 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:23.162491 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:23.165456 26677 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0528 04:37:23.170547 26677 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0528 04:37:23.170938 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.171116 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.171917 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.172678 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.172747 26677 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0528 04:37:23.172954 26677 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0528 04:37:23.173078 26677 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0528 04:37:23.176240 26677 target.go:116] Terraform output is in /tmp/test1237746629/out --- PASS: TestNTHIMDSProcessor (0.20s) === RUN TestNTHIMDSProcessorIRSA I0528 04:37:23.178471 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:23.178564 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:23.178595 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:23.178627 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:23.178651 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:23.178686 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:23.178741 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:23.178758 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:23.178780 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:23.178799 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:23.198009 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:23.199942 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.199999 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.237755 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:23.355910 26677 executor.go:111] Tasks: 0 done / 95 total; 48 can run I0528 04:37:23.356738 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:23.356774 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:23.356866 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:23.358718 26677 executor.go:111] Tasks: 48 done / 95 total; 22 can run I0528 04:37:23.362747 26677 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0528 04:37:23.363005 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.363140 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.364053 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.365122 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.365209 26677 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0528 04:37:23.365457 26677 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0528 04:37:23.365598 26677 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0528 04:37:23.372819 26677 target.go:116] Terraform output is in /tmp/test3536029573/out --- PASS: TestNTHIMDSProcessorIRSA (0.20s) === RUN TestCustomIRSA I0528 04:37:23.375302 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:23.375390 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:23.375420 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:23.375454 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:23.375479 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:23.375513 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:23.375571 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:23.375587 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:23.375610 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:23.375628 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:23.392626 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:23.394614 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.394673 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.433264 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:23.863117 26677 executor.go:111] Tasks: 0 done / 100 total; 50 can run I0528 04:37:23.863966 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:23.864003 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:23.864100 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:23.865316 26677 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0528 04:37:23.869289 26677 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0528 04:37:23.869547 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.869705 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.870426 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.871109 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.871170 26677 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0528 04:37:23.871343 26677 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0528 04:37:23.871493 26677 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0528 04:37:23.878796 26677 target.go:116] Terraform output is in /tmp/test1357844729/out --- PASS: TestCustomIRSA (0.51s) === RUN TestClusterNameDigit I0528 04:37:23.882857 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:23.882963 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:23.882993 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:23.883025 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:23.883050 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:23.883079 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:23.883136 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:23.883155 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:23.883177 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:23.883193 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:23.901522 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:23.903474 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.903530 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:23.941033 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:24.016603 26677 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0528 04:37:24.017052 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:24.017077 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:24.017166 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:24.018938 26677 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0528 04:37:24.025390 26677 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0528 04:37:24.025858 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.027504 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.029885 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.030179 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.030275 26677 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0528 04:37:24.030519 26677 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0528 04:37:24.030640 26677 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0528 04:37:24.033699 26677 target.go:116] Terraform output is in /tmp/test4140863780/out --- PASS: TestClusterNameDigit (0.15s) === RUN TestCASPriorityExpander I0528 04:37:24.036301 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:24.036383 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:24.036411 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:24.036441 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:24.036465 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:24.036498 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:24.036552 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:24.036568 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:24.036592 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:24.036610 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:24.056745 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:37:24.058835 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.058891 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.058925 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:24.058956 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:37:25.465999 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:25.629464 26677 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0528 04:37:25.629668 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:25.629685 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:25.629752 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:25.634380 26677 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0528 04:37:25.639406 26677 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0528 04:37:25.639821 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.639954 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.640174 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.640412 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.641043 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.641185 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.641267 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.642831 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.642913 26677 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0528 04:37:25.643277 26677 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0528 04:37:25.643593 26677 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0528 04:37:25.648177 26677 target.go:116] Terraform output is in /tmp/test4024260938/out --- PASS: TestCASPriorityExpander (1.61s) === RUN TestCASPriorityExpanderCustom I0528 04:37:25.651364 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:25.651466 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:25.651497 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:25.651530 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:25.651567 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:25.651639 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:25.651720 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:25.651743 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:25.651776 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:25.651798 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:25.671753 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0528 04:37:25.674233 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.674312 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.674362 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.674402 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:37:25.714778 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:25.893939 26677 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0528 04:37:25.894715 26677 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0528 04:37:25.894740 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:25.894847 26677 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0528 04:37:25.897517 26677 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0528 04:37:25.903234 26677 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0528 04:37:25.903698 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.903852 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.904101 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.904377 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.904930 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.905676 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.905896 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.906332 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.906442 26677 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0528 04:37:25.906780 26677 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0528 04:37:25.907041 26677 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0528 04:37:25.916589 26677 target.go:116] Terraform output is in /tmp/test1166052202/out --- PASS: TestCASPriorityExpanderCustom (0.27s) === RUN TestLifecycleMinimalAWS I0528 04:37:25.921709 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:25.921847 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:25.921890 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:25.921940 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:25.921981 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:25.922028 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:25.922070 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:25.922099 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:25.922139 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:25.922167 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:25.922221 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0528 04:37:25.955403 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:25.957935 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:25.958019 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:37:25.997325 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:26.086396 26677 iamrole.go:89] ListRoles: { } I0528 04:37:26.087250 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:26.087592 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0528 04:37:26.087702 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0528 04:37:26.087705 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.087809 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.087865 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:26.087861 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0528 04:37:26.087953 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:26.088016 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:26.088021 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0528 04:37:26.088054 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:26.088104 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.088084 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0528 04:37:26.088145 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:26.088372 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.088534 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.088700 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:26.088822 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:26.088878 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.089007 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:26.089083 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:26.089110 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0528 04:37:26.089179 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:26.089664 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:26.089754 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:26.089817 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:26.089807 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.089865 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:26.089893 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:26.089986 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:26.089941 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0528 04:37:26.089997 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0528 04:37:26.090078 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:26.090107 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0528 04:37:26.090156 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:26.090212 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:26.090249 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:26.090304 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:26.090333 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.090430 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:26.090466 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.090667 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.090665 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0528 04:37:26.090797 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:26.090843 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:26.090876 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:26.090927 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.091083 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.091147 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0528 04:37:26.091314 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:26.091398 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:26.091438 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:26.091482 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:26.091323 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:26.094253 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:26.094405 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.094574 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.094656 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.095398 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" }] }] } I0528 04:37:26.095686 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.095908 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.096012 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.096169 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.096261 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.096475 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.096584 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.096700 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I0528 04:37:26.096864 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:26.096925 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.097078 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:26.097149 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.097416 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.097563 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.098001 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.098089 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:26.098165 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.098238 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.098852 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:26.099029 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:26.099075 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:26.100090 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:26.100302 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:26.100399 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:26.100431 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:26.100461 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:26.100502 26677 dns.go:232] Pre-creating DNS records I0528 04:37:26.100519 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:26.103784 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:26.105490 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.105546 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0528 04:37:26.142996 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:26.211474 26677 iamrole.go:89] ListRoles: { } I0528 04:37:26.212370 26677 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0528 04:37:26.212764 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.212897 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.212978 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:26.213009 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.213131 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:26.213258 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.213345 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:26.213374 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:26.213420 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.214270 26677 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0528 04:37:26.214435 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:26.214558 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:26.214480 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0528 04:37:26.214600 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0528 04:37:26.214717 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:26.214755 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0528 04:37:26.214831 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.214930 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:26.214959 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.215034 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.215080 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:26.215123 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:26.217302 26677 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0528 04:37:26.217421 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.217546 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.217621 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.217813 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.217965 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.218052 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.218136 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:26.219346 26677 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0528 04:37:26.219473 26677 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0528 04:37:26.219544 26677 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0528 04:37:26.221082 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:26.221115 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:26.221136 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:26.221154 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:26.221173 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:26.221192 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:26.221210 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:26.221231 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:26.221251 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:26.221271 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:26.221294 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:26.221312 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:26.221330 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:26.221349 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:26.221366 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:26.221387 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:26.221406 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:26.221431 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:26.221450 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:26.221468 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:26.221485 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:26.221505 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:26.221524 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:26.221544 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:26.222312 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:26.222360 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:26.222370 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:26.222392 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222420 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0528 04:37:26.222450 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222481 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222507 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222536 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222560 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222580 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0528 04:37:26.222605 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222633 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:26.222653 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0528 04:37:26.222679 26677 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0528 04:37:26.222697 26677 api.go:53] PageSize not implemented I0528 04:37:26.222710 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:26.222725 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:26.222736 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:26.222752 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:26.222763 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0528 04:37:26.222803 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:26.222829 26677 iamrole.go:89] ListRoles: { } I0528 04:37:26.222872 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:26.222891 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:26.222916 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:26.222945 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:26.222965 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:26.223231 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:26.223293 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I0528 04:37:26.223365 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0528 04:37:26.223280 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:26.223277 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok W0528 04:37:26.223412 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:26.223399 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:26.223441 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0528 04:37:26.223430 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:26.223481 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:26.223520 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0528 04:37:26.223542 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:26.223573 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0528 04:37:26.223613 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0528 04:37:26.223650 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:26.223676 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0528 04:37:26.223783 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:26.223790 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:26.223830 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0528 04:37:26.223860 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:26.223839 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:26.223869 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:26.223888 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0528 04:37:26.223906 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0528 04:37:26.223927 26677 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:26.223934 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:26.223948 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0528 04:37:26.223980 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:26.224022 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:26.224060 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:26.232079 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.31s) === RUN TestLifecycleMinimalOpenstack I0528 04:37:26.232255 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0528 04:37:26.238601 26677 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0528 04:37:26.422255 26677 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0528 04:37:26.422301 26677 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0528 04:37:26.432164 26677 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0528 04:37:26.435151 26677 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0528 04:37:26.443880 26677 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0528 04:37:26.446209 26677 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/05/28 04:37:26 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:26 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:26 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0528 04:37:26.448211 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:26.448407 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:26.448547 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0528 04:37:27.452866 26677 executor.go:111] Tasks: 90 done / 90 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0528 04:37:27.576318 26677 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0528 04:37:27.576359 26677 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0528 04:37:27.591043 26677 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0528 04:37:27.593614 26677 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0528 04:37:27.597644 26677 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0528 04:37:27.598641 26677 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/05/28 04:37:27 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:27 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:27 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0528 04:37:27.600564 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:27.600612 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:27.600816 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0528 04:37:27.601902 26677 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0528 04:37:27.602792 26677 delete_cluster.go:128] Looking for cloud resources to delete 2023/05/28 04:37:27 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) SecurityGroup:890c8818-75f6-433d-a02a-ccb7d23324f8 ok SecurityGroup:5388e860-05ce-4235-895d-71ce9673b3c6 ok Port:443d3329-553e-451e-8622-a1047f26bd6e ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Network:8cf73d68-0ea2-49bc-bd5e-57cc4af1efed ok Instance:c7335602-79c2-4884-9c39-86df4b9e6d2b ok ServerGroup:51d5fb39-3df6-4768-95a1-0ab0da781f5d ok Instance:b113f386-cab9-42da-afba-82f084415b56 ok Subnet:f406b220-bfc5-4d6e-93ba-670b4e873442 ok Port:8c4a140b-12c4-4849-b982-add09de8eede ok Port:c99ced4e-0477-4dd8-8808-40c6248893b9 ok ServerGroup:211c692e-41fd-4591-a3b4-b347102f9656 ok SecurityGroup:f72a5334-9327-4d3e-9438-e1db1e629a59 ok Instance:ec065002-8097-4a40-8332-59ada43bc807 ok --- PASS: TestLifecycleMinimalOpenstack (2.47s) === RUN TestLifecycleMinimalGCE I0528 04:37:28.707123 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0528 04:37:28.708369 26677 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:28.935970 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:28.936075 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:28.936198 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0528 04:37:28.936231 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0528 04:37:28.936258 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0528 04:37:28.936354 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:28.938005 26677 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0528 04:37:28.940474 26677 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0528 04:37:28.940551 26677 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0528 04:37:28.945674 26677 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0528 04:37:28.948205 26677 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0528 04:37:28.948410 26677 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0528 04:37:28.948511 26677 dns.go:232] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0528 04:37:29.188637 26677 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0528 04:37:29.188736 26677 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0528 04:37:29.188851 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0528 04:37:29.188883 26677 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0528 04:37:29.188912 26677 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0528 04:37:29.189000 26677 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0528 04:37:29.190633 26677 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0528 04:37:29.192857 26677 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0528 04:37:29.197834 26677 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0528 04:37:29.199573 26677 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0528 04:37:29.199671 26677 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0528 04:37:29.201109 26677 delete_cluster.go:128] Looking for cloud resources to delete I0528 04:37:29.201158 26677 gce.go:99] Scanning zones: [us-test1-a] I0528 04:37:29.201511 26677 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0528 04:37:29.201587 26677 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok Disk:a-etcd-main-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok Disk:a-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1685248648 ok InstanceTemplate:nodes-minimal-gce-example-com-1685248648 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.50s) === RUN TestLifecycleFloatingIPOpenstack I0528 04:37:29.202843 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0528 04:37:29.208328 26677 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0528 04:37:29.384190 26677 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0528 04:37:29.384231 26677 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0528 04:37:29.394020 26677 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0528 04:37:29.404154 26677 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0528 04:37:29.410281 26677 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0528 04:37:29.412295 26677 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/05/28 04:37:29 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:29 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:29 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0528 04:37:29.414001 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:29.414280 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:29.414530 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0528 04:37:30.420208 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0528 04:37:30.585147 26677 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0528 04:37:30.585184 26677 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0528 04:37:30.596527 26677 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0528 04:37:30.603459 26677 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0528 04:37:30.609759 26677 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0528 04:37:30.610987 26677 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/05/28 04:37:30 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:30 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/05/28 04:37:30 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0528 04:37:30.613034 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:30.613130 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0528 04:37:30.613561 26677 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0528 04:37:30.615634 26677 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0528 04:37:30.617115 26677 delete_cluster.go:128] Looking for cloud resources to delete 2023/05/28 04:37:30 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) Port:7a27cbb2-32af-41b2-bb3e-ba4106338980 ok Instance:6f91d7ef-59ef-45e6-837d-1fc871b98262 ok SecurityGroup:8eb75a38-79bb-4380-9fb5-f25415500ac8 ok Port:2b516f31-e83c-4e1d-b2b6-0c5485a18189 ok Instance:0cf1caad-19b4-4a74-93eb-c6261a3f5b35 ok Subnet:313be472-a26f-4fc7-869f-2e475a5244ad ok ServerGroup:55081a6e-efc2-4b9f-a351-e13ffee2992d ok SecurityGroup:de1d3485-c053-4458-a9c7-644caf0a334c ok Port:76fc03e2-36fd-4b7b-ba47-96288836add2 ok Instance:358e7685-fb9b-45f3-b9af-42c989a215c7 ok ServerGroup:6f1cc472-441e-407e-931d-466858d21035 ok SecurityGroup:d80049b7-563d-4067-a4b4-8580834742a2 ok Network:a444246f-77c9-444a-8414-038d2e810865 ok Port:4fbf101b-074b-48c8-b3d6-bc12189a8ea3 ok Router-IF:313be472-a26f-4fc7-869f-2e475a5244ad ok Router:d7499832-4f0a-46ee-a0f5-d6b6b7ef4aed ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok --- PASS: TestLifecycleFloatingIPOpenstack (2.53s) === RUN TestLifecyclePrivateCalico I0528 04:37:31.729575 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:31.729718 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:31.729766 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:31.729823 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:31.729868 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:31.729917 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:31.729963 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:31.729996 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:31.730035 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:31.730063 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:31.730117 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0528 04:37:31.767908 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:31.772590 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:31.772728 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:31.772823 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:31.813667 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.182143 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.183515 26677 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0528 04:37:32.183751 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0528 04:37:32.183870 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0528 04:37:32.183839 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.184062 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.184172 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:32.184225 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.184284 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0528 04:37:32.184351 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0528 04:37:32.184383 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:32.184506 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:32.184543 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:32.184624 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0528 04:37:32.184668 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:32.184698 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.184709 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" }] } I0528 04:37:32.184843 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.185041 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0528 04:37:32.185126 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:32.185157 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:32.185186 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:32.185228 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.185259 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0528 04:37:32.185414 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:32.185552 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:32.186276 26677 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0528 04:37:32.186558 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0528 04:37:32.186689 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0528 04:37:32.186702 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0528 04:37:32.186780 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0528 04:37:32.186702 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.186870 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0528 04:37:32.186926 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0528 04:37:32.186876 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatecalico-exa-hocohm", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatecalico-exa-hocohm" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0528 04:37:32.186967 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0528 04:37:32.186984 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0528 04:37:32.187017 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0528 04:37:32.186962 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:32.186995 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0528 04:37:32.187147 26677 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0528 04:37:32.187175 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.187268 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:32.187312 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.187330 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0528 04:37:32.187349 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.187478 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:32.187526 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.187617 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:32.187682 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.187730 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:32.187793 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.187840 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:32.187874 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:32.187901 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I0528 04:37:32.187944 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.188031 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:32.187847 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0528 04:37:32.188080 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.188111 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.188178 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0528 04:37:32.188292 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0528 04:37:32.188400 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0528 04:37:32.188498 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.188590 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:32.188625 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.188659 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:32.188706 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:32.188756 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.188782 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0528 04:37:32.188834 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:32.188859 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.188882 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0528 04:37:32.188957 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0528 04:37:32.188988 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.189027 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.189054 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:32.189079 26677 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] } I0528 04:37:32.189138 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:32.189185 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:32.189254 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0528 04:37:32.189362 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0528 04:37:32.189479 26677 api.go:53] PageSize not implemented I0528 04:37:32.189501 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.189516 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:32.189608 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.189618 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.189594 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0528 04:37:32.189722 26677 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0528 04:37:32.189770 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:32.189738 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatecalico-exa-hocohm", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], Type: "network" } I0528 04:37:32.189835 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.189793 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.189906 26677 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0528 04:37:32.189905 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion.privatecalico.example.com" }] }] } I0528 04:37:32.189890 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1", Port: 22, Protocol: "TCP" } I0528 04:37:32.189960 26677 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } W0528 04:37:32.190045 26677 api.go:53] PageSize not implemented I0528 04:37:32.190061 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.190179 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.190384 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0528 04:37:32.190547 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.190665 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.190743 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.190875 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.190973 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.191056 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.191127 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.191215 26677 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0528 04:37:32.191249 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0528 04:37:32.191377 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:32.191420 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:32.191455 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:32.191548 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0528 04:37:32.191649 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0528 04:37:32.191735 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.191858 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.191981 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:32.192026 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192150 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0528 04:37:32.192278 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192395 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192513 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192632 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" }] }] } I0528 04:37:32.192751 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192870 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.192993 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.193110 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.193226 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:32.193312 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.193447 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.193577 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0528 04:37:32.193624 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0528 04:37:32.193659 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.193778 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.193875 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.193983 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0528 04:37:32.194105 26677 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0528 04:37:32.194173 26677 api.go:53] PageSize not implemented I0528 04:37:32.194189 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.194234 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.194239 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.194300 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.194318 26677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } W0528 04:37:32.194403 26677 api.go:53] PageSize not implemented I0528 04:37:32.194424 26677 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0528 04:37:32.194448 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.194488 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:32.194513 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.194538 26677 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0528 04:37:32.198861 26677 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0528 04:37:32.199037 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.199226 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.199344 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.199417 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.199558 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.199639 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0528 04:37:32.202212 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0528 04:37:32.202455 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:32.202556 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:32.202601 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.202633 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.202690 26677 dns.go:232] Pre-creating DNS records I0528 04:37:32.202712 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.211359 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:32.213536 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.213600 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.213639 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.251611 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.629918 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.632802 26677 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0528 04:37:32.633122 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.633417 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:32.633373 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.633593 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:32.633718 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.633799 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:32.633820 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:32.633855 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.633895 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.635312 26677 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0528 04:37:32.635417 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0528 04:37:32.635492 26677 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0528 04:37:32.635414 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.635521 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0528 04:37:32.635574 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0528 04:37:32.635675 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0528 04:37:32.635749 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0528 04:37:32.635787 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0528 04:37:32.635753 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.635861 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.635951 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0528 04:37:32.636076 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.636119 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.636249 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.636346 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0528 04:37:32.636453 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.636494 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.636555 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.636650 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.636679 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0528 04:37:32.636773 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.636814 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.636901 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:32.636951 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0528 04:37:32.637047 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0528 04:37:32.637227 26677 api.go:53] PageSize not implemented I0528 04:37:32.637195 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.637204 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:32.637263 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.637247 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.637280 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0528 04:37:32.637307 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.637314 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0528 04:37:32.637336 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0528 04:37:32.637370 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0528 04:37:32.637398 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0528 04:37:32.637334 26677 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } W0528 04:37:32.637454 26677 api.go:53] PageSize not implemented I0528 04:37:32.637468 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.637646 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.637752 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.637851 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.637911 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:32.637942 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0528 04:37:32.637965 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0528 04:37:32.638007 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.638056 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.638117 26677 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0528 04:37:32.638161 26677 api.go:53] PageSize not implemented I0528 04:37:32.638164 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.638208 26677 dnsname.go:118] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0528 04:37:32.638186 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.638205 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.638256 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0528 04:37:32.638384 26677 api.go:53] PageSize not implemented I0528 04:37:32.638440 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.638609 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.641803 26677 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0528 04:37:32.641865 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.641953 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.642056 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.642172 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.642258 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.643971 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0528 04:37:32.644032 26677 api.go:53] PageSize not implemented I0528 04:37:32.644051 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.644177 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:32.644284 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:32.646707 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:32.646750 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:32.646780 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:32.646805 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0528 04:37:32.646832 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:32.646856 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:32.646878 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:32.646901 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:32.646927 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:32.646953 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:32.646980 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:32.647008 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0528 04:37:32.647036 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:32.647062 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:32.647088 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:32.647111 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:32.647136 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:32.647158 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:32.647183 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:32.647208 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:32.647234 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:32.647259 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:32.647283 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.647309 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:32.647332 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:32.647358 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:32.647381 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:32.647404 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:32.647429 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:32.647456 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:32.647482 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:32.648564 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:32.648612 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:32.648627 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:32.648654 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648695 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0528 04:37:32.648741 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648786 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648816 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648852 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648882 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648915 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0528 04:37:32.648955 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.648992 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:32.649011 26677 natgateway.go:117] DescribeNatGateways: { } I0528 04:37:32.649035 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0528 04:37:32.649062 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0528 04:37:32.649098 26677 api.go:53] PageSize not implemented I0528 04:37:32.649109 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0528 04:37:32.649135 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:32.649154 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.649163 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0528 04:37:32.649189 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:32.649204 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:32.649215 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" ] } I0528 04:37:32.649256 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:32.649283 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.649322 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.649344 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.649374 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:32.649408 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:32.649436 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0528 04:37:32.649462 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:32.649487 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0528 04:37:32.649918 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0528 04:37:32.649964 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0528 04:37:32.649984 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:32.649998 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0528 04:37:32.650030 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:bastion.privatecalico.example.com ok I0528 04:37:32.650084 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0528 04:37:32.650009 26677 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1 ok load-balancer:api-privatecalico-example-0uch4k ok I0528 04:37:32.650078 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.650108 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0528 04:37:32.650137 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0528 04:37:32.650152 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4 ok I0528 04:37:32.650069 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0528 04:37:32.650140 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0528 04:37:32.650304 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.650324 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:32.650356 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0528 04:37:32.650405 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.650443 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0528 04:37:32.650518 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.650532 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-3 ok I0528 04:37:32.650562 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.650594 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } W0528 04:37:32.650710 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.650725 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0528 04:37:32.650755 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0528 04:37:32.650790 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:32.650835 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0528 04:37:32.650861 26677 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0528 04:37:32.650943 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0528 04:37:32.650980 26677 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0528 04:37:32.650961 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0528 04:37:32.650993 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } subnet:subnet-1 ok I0528 04:37:32.651015 26677 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0528 04:37:32.651018 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0528 04:37:32.651038 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0528 04:37:32.651047 26677 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I0528 04:37:32.651071 26677 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com elastic-ip:eipalloc-1 ok I0528 04:37:32.651081 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0528 04:37:32.651100 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0528 04:37:32.651094 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.651122 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0528 04:37:32.651149 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0528 04:37:32.651157 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0528 04:37:32.651175 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0528 04:37:32.651131 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 } ] } W0528 04:37:32.651246 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.651257 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0528 04:37:32.651303 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0528 04:37:32.651346 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:32.651438 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:32.651508 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:32.651720 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.92s) === RUN TestLifecyclePrivateKopeio I0528 04:37:32.651856 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:32.651953 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:32.651980 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:32.652008 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:32.652037 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:32.652075 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:32.652103 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:32.652123 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:32.652148 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:32.652168 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:32.652203 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0528 04:37:32.673044 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:32.675519 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.675598 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.675651 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.713656 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.796906 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.798580 26677 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0528 04:37:32.798871 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.798953 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0528 04:37:32.799016 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0528 04:37:32.799022 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.799046 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:32.799091 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0528 04:37:32.799136 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:32.799185 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0528 04:37:32.799216 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0528 04:37:32.799356 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:32.799384 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:32.799381 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0528 04:37:32.799441 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0528 04:37:32.799411 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:32.799499 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.799525 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.799631 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:32.799720 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:32.799748 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.799842 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:32.799917 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:32.799944 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.800041 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.800101 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:32.800701 26677 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0528 04:37:32.800840 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0528 04:37:32.800793 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0528 04:37:32.800917 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0528 04:37:32.800916 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.800960 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0528 04:37:32.801028 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0528 04:37:32.801086 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0528 04:37:32.801069 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatekopeio-exa-d8ef8e", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatekopeio-exa-d8ef8e" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0528 04:37:32.801127 26677 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0528 04:37:32.801073 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.801147 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0528 04:37:32.801140 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0528 04:37:32.801166 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0528 04:37:32.801228 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0528 04:37:32.801280 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0528 04:37:32.801309 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.801367 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:32.801402 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.801426 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.801484 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0528 04:37:32.801555 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:32.801562 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.801783 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.801766 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:32.801854 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:32.801936 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:32.801943 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.802088 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:32.802152 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.802210 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.802249 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.802335 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0528 04:37:32.802438 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.802519 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.802432 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0528 04:37:32.802591 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0528 04:37:32.802691 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0528 04:37:32.802600 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:32.802678 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:32.802761 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.802823 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.802903 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:32.802942 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:32.802958 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0528 04:37:32.802979 26677 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0528 04:37:32.803038 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0528 04:37:32.803148 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:32.803187 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0528 04:37:32.803241 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0528 04:37:32.803278 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.803361 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:32.803402 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.803455 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0528 04:37:32.803495 26677 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0528 04:37:32.803580 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:32.803621 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:32.803659 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:32.803710 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.803753 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:32.803793 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:32.803822 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:32.803880 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:32.803909 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.803933 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:32.803958 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:32.803984 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0528 04:37:32.804056 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0528 04:37:32.804124 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.804188 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.804242 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0528 04:37:32.804295 26677 api.go:53] PageSize not implemented I0528 04:37:32.804359 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:32.804330 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0528 04:37:32.804390 26677 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } W0528 04:37:32.804411 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.804418 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.804442 26677 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0528 04:37:32.804417 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.804493 26677 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" }] } I0528 04:37:32.804555 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.804464 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:32.804586 26677 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0528 04:37:32.804522 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatekopeio-exa-d8ef8e", SubnetMappings: [{ SubnetId: "subnet-4" },{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }], Type: "network" } I0528 04:37:32.804634 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0528 04:37:32.804637 26677 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0528 04:37:32.804677 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0528 04:37:32.804716 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.804748 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1", Port: 22, Protocol: "TCP" } W0528 04:37:32.804801 26677 api.go:53] PageSize not implemented I0528 04:37:32.804816 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.804870 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.805090 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.805209 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:32.805327 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0528 04:37:32.805380 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.805454 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.805534 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0528 04:37:32.805640 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.805764 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.805857 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0528 04:37:32.805897 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.805938 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:32.805960 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.806004 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0528 04:37:32.806034 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.806116 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.806261 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0528 04:37:32.806302 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.8.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.806439 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0528 04:37:32.806568 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0528 04:37:32.806698 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.806815 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0528 04:37:32.806855 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0528 04:37:32.806887 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0528 04:37:32.806974 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.807237 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:32.807318 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:32.807405 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:32.807459 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.807536 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0528 04:37:32.807585 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:32.807655 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0528 04:37:32.807677 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:32.807701 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0528 04:37:32.807775 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0528 04:37:32.807802 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0528 04:37:32.807824 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.807898 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.807974 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }] } I0528 04:37:32.808027 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0528 04:37:32.808101 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0528 04:37:32.808172 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.808264 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0528 04:37:32.808347 26677 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0528 04:37:32.808376 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0528 04:37:32.808404 26677 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0528 04:37:32.808426 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0528 04:37:32.808448 26677 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0528 04:37:32.808477 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0528 04:37:32.808515 26677 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0528 04:37:32.808597 26677 api.go:53] PageSize not implemented I0528 04:37:32.808622 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.808634 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:32.808640 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.808734 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.808761 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0528 04:37:32.808803 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0528 04:37:32.808864 26677 api.go:53] PageSize not implemented I0528 04:37:32.808891 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.808913 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0528 04:37:32.811499 26677 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0528 04:37:32.811626 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.811697 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.811774 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.811871 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.813919 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0528 04:37:32.814176 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:32.814281 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:32.814319 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.814350 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.814398 26677 dns.go:232] Pre-creating DNS records I0528 04:37:32.814418 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.820194 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:32.823187 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.823271 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.823329 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.861428 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.944789 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.946226 26677 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0528 04:37:32.946383 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.946566 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.946682 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:32.946732 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:32.946858 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.946974 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:32.947008 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:32.947066 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.947250 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.950848 26677 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0528 04:37:32.950987 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0528 04:37:32.951037 26677 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0528 04:37:32.950987 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0528 04:37:32.951086 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0528 04:37:32.950975 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.951136 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0528 04:37:32.951096 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0528 04:37:32.951174 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0528 04:37:32.951454 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0528 04:37:32.952019 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.952057 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.952161 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0528 04:37:32.952188 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.952270 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:32.952661 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.952833 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I0528 04:37:32.952989 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:32.953043 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.953163 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:32.953255 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.953365 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0528 04:37:32.953493 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0528 04:37:32.953615 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.953733 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.953827 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.953862 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.953917 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.953982 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:32.954078 26677 executor.go:111] Tasks: 76 done / 128 total; 37 can run W0528 04:37:32.954203 26677 api.go:53] PageSize not implemented I0528 04:37:32.954208 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0528 04:37:32.954224 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.954267 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:32.954348 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.954322 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.954369 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0528 04:37:32.954403 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0528 04:37:32.954319 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.954431 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0528 04:37:32.954425 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.954464 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0528 04:37:32.954467 26677 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0528 04:37:32.954494 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0528 04:37:32.954491 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } W0528 04:37:32.954551 26677 api.go:53] PageSize not implemented I0528 04:37:32.954566 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.954577 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.954682 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:32.954798 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.954866 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.955062 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.955253 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:32.955421 26677 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0528 04:37:32.955455 26677 api.go:53] PageSize not implemented I0528 04:37:32.955496 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.955529 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:32.955586 26677 dnsname.go:118] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0528 04:37:32.955604 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.955550 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0528 04:37:32.955640 26677 api.go:53] PageSize not implemented I0528 04:37:32.955664 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:32.955679 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.959435 26677 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0528 04:37:32.959536 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.959686 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.959815 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.959918 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.961597 26677 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0528 04:37:32.961672 26677 api.go:53] PageSize not implemented I0528 04:37:32.961691 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.961807 26677 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0528 04:37:32.961897 26677 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0528 04:37:32.965292 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:32.965350 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:32.965388 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:32.965424 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:32.965457 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:32.965489 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:32.965528 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:32.965566 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:32.965600 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:32.965638 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:32.965674 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0528 04:37:32.965715 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:32.965748 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:32.965779 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:32.965816 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:32.965846 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:32.965883 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0528 04:37:32.965917 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:32.965952 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:32.965985 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:32.966018 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:32.966051 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:32.966086 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:32.966116 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:32.966153 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:32.966188 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0528 04:37:32.966222 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:32.966251 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:32.966282 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:32.966318 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:32.966352 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:32.966385 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:32.968082 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:32.968144 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:32.968161 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:32.968199 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968262 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0528 04:37:32.968339 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968407 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968452 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968508 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968549 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968595 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0528 04:37:32.968656 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968722 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:32.968755 26677 natgateway.go:117] DescribeNatGateways: { } I0528 04:37:32.968791 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0528 04:37:32.968829 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0528 04:37:32.968876 26677 api.go:53] PageSize not implemented I0528 04:37:32.968892 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0528 04:37:32.968930 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:32.968954 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:32.968967 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0528 04:37:32.969008 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:32.969029 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:32.969045 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" ] } I0528 04:37:32.969099 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:32.969142 26677 iamrole.go:89] ListRoles: { } I0528 04:37:32.969191 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.969218 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:32.969258 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:32.969299 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:32.969342 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-3","rtb-2"] } I0528 04:37:32.969382 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0528 04:37:32.969413 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0528 04:37:32.969975 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:32.970046 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok I0528 04:37:32.970058 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0528 04:37:32.970091 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0528 04:37:32.970013 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-config:lt-1 ok route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok keypair:key-1 ok I0528 04:37:32.970129 26677 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0528 04:37:32.970196 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0528 04:37:32.970163 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } volume:vol-1 ok iam-instance-profile:bastions.privatekopeio.example.com ok autoscaling-config:lt-2 ok I0528 04:37:32.970202 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4 ok I0528 04:37:32.970252 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:32.970267 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0528 04:37:32.970283 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1 ok I0528 04:37:32.970319 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0528 04:37:32.970314 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:32.970348 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0528 04:37:32.970393 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0528 04:37:32.970396 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0528 04:37:32.970554 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.970573 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:32.970598 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:32.970631 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0528 04:37:32.970723 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.970739 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-3 ok I0528 04:37:32.970771 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:32.970803 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0528 04:37:32.970916 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.970925 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0528 04:37:32.971010 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:32.971006 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0528 04:37:32.971060 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:32.971102 26677 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0528 04:37:32.971115 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0528 04:37:32.971146 26677 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0528 04:37:32.971168 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0528 04:37:32.971199 26677 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0528 04:37:32.971115 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0528 04:37:32.971224 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:32.971206 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0528 04:37:32.971250 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:32.971269 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0528 04:37:32.971272 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0528 04:37:32.971318 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0528 04:37:32.971329 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } I0528 04:37:32.971354 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com subnet:subnet-4 ok I0528 04:37:32.971365 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0528 04:37:32.971369 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0528 04:37:32.971389 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0528 04:37:32.971446 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0528 04:37:32.971526 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0528 04:37:32.971595 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:32.971686 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:32.971756 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:32.972005 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.32s) === RUN TestLifecycleIPv6 I0528 04:37:32.972166 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:32.972282 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:32.972310 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:32.972340 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:32.972367 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:32.972404 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:32.972433 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:32.972453 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:32.972480 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:32.972501 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:32.972536 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0528 04:37:32.992536 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:32.994698 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:32.994758 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.032559 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.165789 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.167006 26677 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0528 04:37:33.167373 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.167473 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.167536 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:33.167587 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0528 04:37:33.167544 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:33.167720 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.167816 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.167887 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.167927 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I0528 04:37:33.168008 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:33.168063 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.168234 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.168240 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.168450 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.168625 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:33.168659 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:33.168685 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:33.168730 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.168755 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.168839 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:33.168867 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.168936 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:33.169047 26677 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0528 04:37:33.169118 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0528 04:37:33.169119 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.169176 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0528 04:37:33.169183 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.169256 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.169217 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0528 04:37:33.169307 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0528 04:37:33.169384 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:33.169396 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0528 04:37:33.169423 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.169419 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.169484 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.169507 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:33.169591 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0528 04:37:33.169624 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0528 04:37:33.169749 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.169784 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:33.169847 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:33.169877 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.169900 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0528 04:37:33.170010 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:33.170057 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:33.170084 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.170158 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:33.170191 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.170287 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:33.170319 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0528 04:37:33.170391 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.170443 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.170497 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:33.170553 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:33.170579 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0528 04:37:33.170621 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.170648 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0528 04:37:33.170721 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0528 04:37:33.170771 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:33.170798 26677 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I0528 04:37:33.170841 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0528 04:37:33.170720 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.170911 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0528 04:37:33.170951 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:33.171001 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:33.171035 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:33.171059 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0528 04:37:33.171082 26677 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0528 04:37:33.171144 26677 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0528 04:37:33.171206 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.171282 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.171381 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.171515 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-2" } I0528 04:37:33.171635 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.171799 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.171944 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.172074 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.172154 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.172280 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:113::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "Name", Value: "dualstack-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0528 04:37:33.172298 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.172479 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.172499 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.4.0/22", Ipv6CidrBlock: "2001:db8:0:116::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0528 04:37:33.172616 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:33.172599 26677 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.172664 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.172782 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.172864 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.172858 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", Ipv6CidrBlock: "2001:db8:0:112::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0528 04:37:33.172944 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:33.172985 26677 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.173005 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.173105 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.173188 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0528 04:37:33.173187 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", Ipv6CidrBlock: "2001:db8:0:114::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "dualstack-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0528 04:37:33.173264 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.173321 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-3" } I0528 04:37:33.173260 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", Ipv6CidrBlock: "2001:db8:0:111::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0528 04:37:33.173344 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" }] }] } I0528 04:37:33.173410 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.0.0/22", Ipv6CidrBlock: "2001:db8:0:115::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.173430 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.173531 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:33.173561 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.173637 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" }] }] } I0528 04:37:33.173718 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.173798 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:33.173828 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.173910 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.173999 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:33.174030 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:33.174052 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.174129 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:33.174197 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.174273 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.174347 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0528 04:37:33.174398 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0528 04:37:33.174472 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0528 04:37:33.174543 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.174677 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0528 04:37:33.174727 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0528 04:37:33.174797 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0528 04:37:33.174855 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0528 04:37:33.174885 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0528 04:37:33.174922 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0528 04:37:33.174950 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:33.174976 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.175014 26677 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0528 04:37:33.175066 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.175093 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.175145 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.175111 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.175219 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-6" },{ SubnetId: "subnet-2" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0528 04:37:33.175260 26677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:33.175337 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.175346 26677 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.175408 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0528 04:37:33.175436 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:33.175465 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0528 04:37:33.175515 26677 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } W0528 04:37:33.175556 26677 api.go:53] PageSize not implemented I0528 04:37:33.175542 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.175677 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:33.175715 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0528 04:37:33.175751 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.175868 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-5"] }] } I0528 04:37:33.175904 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-5" } I0528 04:37:33.175939 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-6"] }] } I0528 04:37:33.175974 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-6" } I0528 04:37:33.176003 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.176115 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0528 04:37:33.176155 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0528 04:37:33.176185 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.176252 26677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-6"] }] } I0528 04:37:33.176316 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:33.176367 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:33.176353 26677 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-2", SubnetId: "subnet-6", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0528 04:37:33.176394 26677 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0528 04:37:33.176437 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0528 04:37:33.176473 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0528 04:37:33.176504 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.176630 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0528 04:37:33.176669 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0528 04:37:33.176707 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0528 04:37:33.176751 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0528 04:37:33.176789 26677 tags.go:42] CreateTags { Resources: ["subnet-6"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-2" }] } I0528 04:37:33.176858 26677 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0528 04:37:33.176895 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.176942 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.176915 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.176938 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.176957 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.177009 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.177028 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-2", RouteTableId: "rtb-2" } I0528 04:37:33.177030 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.177070 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.177107 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.177088 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.177179 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-3" } I0528 04:37:33.177212 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.177221 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.177331 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.177421 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-2", RouteTableId: "rtb-2" } I0528 04:37:33.177465 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.177572 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-3" } I0528 04:37:33.183918 26677 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0528 04:37:33.184125 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.184212 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.184283 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.184387 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.186467 26677 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0528 04:37:33.186746 26677 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0528 04:37:33.186883 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:33.186939 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.186990 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.187074 26677 dns.go:232] Pre-creating DNS records I0528 04:37:33.187105 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.191265 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0528 04:37:33.193446 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.193501 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.230573 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.356673 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.359252 26677 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0528 04:37:33.359452 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.359581 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:33.359613 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:33.359667 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.359611 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:33.359711 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:33.359891 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.360063 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.360193 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.362963 26677 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0528 04:37:33.363184 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0528 04:37:33.363155 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.363216 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0528 04:37:33.363335 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0528 04:37:33.363291 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0528 04:37:33.363403 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0528 04:37:33.363336 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0528 04:37:33.363517 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.363582 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.363716 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.363817 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0528 04:37:33.363976 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:33.364045 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.364156 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.364218 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.364296 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0528 04:37:33.364397 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.364427 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.365352 26677 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0528 04:37:33.365428 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.365521 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0528 04:37:33.365549 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.365619 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] }] } I0528 04:37:33.365722 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:33.365790 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0528 04:37:33.365870 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0528 04:37:33.365915 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0528 04:37:33.366150 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.366315 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.366499 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.366550 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.366685 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0528 04:37:33.366774 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.366958 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:33.367002 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.367143 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0528 04:37:33.367253 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.367411 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.367508 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.367587 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:33.367624 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0528 04:37:33.367663 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0528 04:37:33.367720 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.367868 26677 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0528 04:37:33.367933 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.367988 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.368058 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.368062 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.368161 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.368182 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.368109 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.368227 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.368290 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0528 04:37:33.368331 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0528 04:37:33.368366 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0528 04:37:33.368252 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } W0528 04:37:33.368460 26677 api.go:53] PageSize not implemented I0528 04:37:33.368456 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.368545 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:33.368602 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.368678 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.368755 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.368856 26677 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0528 04:37:33.368919 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.368916 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.369017 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.368998 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.369041 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.368949 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.369123 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.369164 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.369178 26677 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0528 04:37:33.369176 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.369096 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0528 04:37:33.369212 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.369288 26677 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0528 04:37:33.369303 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:33.369336 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.369305 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.369396 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:33.369421 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.372008 26677 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0528 04:37:33.372117 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.372225 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.372281 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.372369 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.373646 26677 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0528 04:37:33.373776 26677 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0528 04:37:33.373843 26677 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0528 04:37:33.375768 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:33.375804 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:33.375827 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:33.375847 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-2"] }] } I0528 04:37:33.375867 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0528 04:37:33.375887 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:33.375908 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0528 04:37:33.375929 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:33.375949 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:33.375967 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:33.375987 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:33.376011 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:33.376035 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:33.376058 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:33.376079 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0528 04:37:33.376100 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:33.376123 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:33.376142 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:33.376164 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:33.376183 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:33.376205 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0528 04:37:33.376227 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:33.376247 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:33.376268 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:33.376288 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:33.376310 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:33.376330 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:33.376353 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:33.376374 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0528 04:37:33.376395 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0528 04:37:33.376416 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0528 04:37:33.376437 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:33.376456 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:33.376475 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:33.376498 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:33.376520 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:33.376541 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:33.377679 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:33.377718 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:33.377729 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:33.377753 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377784 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0528 04:37:33.377820 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377855 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377884 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377912 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377940 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.377969 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0528 04:37:33.378010 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.378058 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:33.378078 26677 natgateway.go:117] DescribeNatGateways: { } I0528 04:37:33.378104 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0528 04:37:33.378129 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0528 04:37:33.378160 26677 api.go:53] PageSize not implemented I0528 04:37:33.378171 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.378187 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.378196 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0528 04:37:33.378218 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:33.378231 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:33.378241 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0528 04:37:33.378289 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:33.378315 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.378357 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.378375 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.378399 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:33.378425 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:33.378457 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1","rtb-3"] } I0528 04:37:33.378482 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0528 04:37:33.378504 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0528 04:37:33.378524 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:33.378541 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0528 04:37:33.378946 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.minimal-ipv6.example.com ok I0528 04:37:33.378985 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok W0528 04:37:33.379067 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:33.379026 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.379080 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0528 04:37:33.379123 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:33.379119 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.379143 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-6" } iam-instance-profile:nodes.minimal-ipv6.example.com ok subnet:subnet-6 ok I0528 04:37:33.379167 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:33.379132 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0528 04:37:33.379199 26677 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok nat-gateway:nat-1 ok I0528 04:37:33.379112 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.379227 26677 egressonlyinternetgateways.go:171] DeleteEgressOnlyInternetGateway: { EgressOnlyInternetGatewayId: "eigw-2" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok egress-only-internet-gateway:eigw-2 ok I0528 04:37:33.379202 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.379257 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.379286 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0528 04:37:33.379290 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:33.379261 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok autoscaling-config:lt-1 ok I0528 04:37:33.379405 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0528 04:37:33.379493 26677 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-2" } nat-gateway:nat-2 ok I0528 04:37:33.379559 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:33.379631 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0528 04:37:33.379677 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:33.379733 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0528 04:37:33.380040 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:33.380065 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0528 04:37:33.380099 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:33.380158 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0528 04:37:33.380275 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0528 04:37:33.380330 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.380336 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:33.380380 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0528 04:37:33.380418 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0528 04:37:33.380451 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0528 04:37:33.380426 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:33.380485 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.380523 26677 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0528 04:37:33.380538 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0528 04:37:33.380526 26677 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-2" } I0528 04:37:33.380564 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok elastic-ip:eipalloc-2 ok I0528 04:37:33.380592 26677 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0528 04:37:33.380648 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-5" } subnet:subnet-5 ok I0528 04:37:33.380736 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0528 04:37:33.380791 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0528 04:37:33.380863 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:33.380936 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:33.381203 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.41s) === RUN TestLifecycleSharedVPC I0528 04:37:33.381394 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:33.381534 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:33.381572 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:33.381646 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:33.381687 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:33.381735 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:33.381775 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:33.381805 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:33.381844 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:33.381872 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:33.381924 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0528 04:37:33.407103 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:33.410095 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.410183 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.447874 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.543873 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.544793 26677 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0528 04:37:33.545094 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:33.545120 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:33.545174 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:33.545198 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:33.545231 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] }] } I0528 04:37:33.545280 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0528 04:37:33.545358 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.545461 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:33.545375 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0528 04:37:33.545497 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.548196 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0528 04:37:33.548184 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:33.548437 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0528 04:37:33.548674 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0528 04:37:33.548952 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.549155 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:33.552016 26677 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0528 04:37:33.552159 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0528 04:37:33.552342 26677 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0528 04:37:33.552393 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0528 04:37:33.552495 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0528 04:37:33.552414 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:33.552564 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:33.552648 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:33.552686 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.552696 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0528 04:37:33.552714 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.552782 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:33.552851 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.552868 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0528 04:37:33.552922 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:33.552889 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0528 04:37:33.552950 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] } I0528 04:37:33.552989 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0528 04:37:33.553011 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0528 04:37:33.553248 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:33.553336 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:33.553386 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.553452 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:33.553522 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.554141 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0528 04:37:33.555238 26677 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0528 04:37:33.555381 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.555558 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.555652 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.555918 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.556112 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" }] }] } I0528 04:37:33.556300 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.556491 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0528 04:37:33.556623 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.556807 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.556931 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.557012 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.557143 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.557249 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.557335 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.557406 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.557566 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.557774 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.557892 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0528 04:37:33.557944 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.558062 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0528 04:37:33.558107 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.558191 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.558273 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0528 04:37:33.558389 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:33.558416 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:33.559606 26677 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0528 04:37:33.559811 26677 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0528 04:37:33.559935 26677 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0528 04:37:33.559977 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.560010 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.560064 26677 dns.go:232] Pre-creating DNS records I0528 04:37:33.560090 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.564842 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:33.567397 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.567471 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.605802 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.704152 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.705596 26677 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0528 04:37:33.705995 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I0528 04:37:33.706193 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.706333 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:33.706454 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:33.706447 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:33.706525 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:33.706578 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:33.709471 26677 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0528 04:37:33.709750 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:33.709866 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0528 04:37:33.709932 26677 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0528 04:37:33.709878 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0528 04:37:33.711141 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.711345 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.711519 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.711696 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.711994 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0528 04:37:33.712200 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.717525 26677 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0528 04:37:33.717745 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.717899 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.718137 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.718409 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.718668 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.718841 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.718995 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.721194 26677 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0528 04:37:33.721418 26677 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0528 04:37:33.721551 26677 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0528 04:37:33.724101 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:33.724163 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:33.724197 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:33.724231 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:33.724260 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:33.724291 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:33.724343 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:33.724382 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:33.724425 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:33.724459 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:33.724489 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:33.724525 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:33.724556 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:33.724587 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:33.724621 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:33.724656 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:33.724690 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:33.724721 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:33.724751 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:33.724785 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:33.724821 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:33.726141 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:33.726207 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:33.726227 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:33.726267 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726316 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0528 04:37:33.726369 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726431 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726470 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726509 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726550 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726587 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0528 04:37:33.726631 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726684 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0528 04:37:33.726717 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0528 04:37:33.726764 26677 api.go:53] PageSize not implemented I0528 04:37:33.726785 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.726810 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.726829 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:33.726852 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:33.726871 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0528 04:37:33.726922 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:33.726958 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.727005 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.727032 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.727075 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:33.727114 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:33.727144 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.727533 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:33.727604 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:33.727584 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } autoscaling-group:nodes.sharedvpc.example.com ok I0528 04:37:33.727671 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0528 04:37:33.727656 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0528 04:37:33.727735 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0528 04:37:33.727835 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0528 04:37:33.727773 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0528 04:37:33.727913 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:33.727716 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0528 04:37:33.727940 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0528 04:37:33.728070 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0528 04:37:33.728170 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.728268 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0528 04:37:33.728596 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:33.728622 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:33.728727 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0528 04:37:33.728846 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0528 04:37:33.728808 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:33.728879 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0528 04:37:33.728950 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0528 04:37:33.728990 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0528 04:37:33.729033 26677 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0528 04:37:33.729046 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0528 04:37:33.729074 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0528 04:37:33.729123 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:33.729363 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.35s) === RUN TestLifecycleComplex I0528 04:37:33.729535 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:33.729664 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:33.729700 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:33.729739 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:33.729776 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:33.729821 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:33.729861 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:33.729892 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:33.729928 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:33.729953 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:33.730002 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0528 04:37:33.765572 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:33.768863 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.768938 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.807529 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.973394 26677 iamrole.go:89] ListRoles: { } I0528 04:37:33.974667 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:33.974908 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.975086 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0528 04:37:33.975182 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:33.975188 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } W0528 04:37:33.975011 26677 api.go:53] PageSize not implemented I0528 04:37:33.975253 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0528 04:37:33.975242 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:33.975306 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0528 04:37:33.975314 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:33.975346 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0528 04:37:33.975361 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.975522 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:33.975620 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:33.975686 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:33.975738 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.975854 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.975975 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:33.976009 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:33.977160 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0528 04:37:33.977166 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:33.977534 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:33.977775 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:33.977846 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:33.977885 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:33.978045 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.978784 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:33.978917 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:33.978978 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.979193 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.979209 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:33.979230 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0528 04:37:33.979339 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.979512 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0528 04:37:33.979445 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0528 04:37:33.979519 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:33.979682 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:33.979697 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.979812 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0528 04:37:33.979864 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:33.979955 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:33.979977 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.979932 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.980401 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0528 04:37:33.980390 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.980471 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:33.980472 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:33.980499 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:33.980544 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:33.980575 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.980642 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:33.980711 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:33.981239 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.981322 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:33.981385 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.981458 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:33.981530 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:33.981530 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.981633 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0528 04:37:33.981733 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.981758 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0528 04:37:33.981796 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.981861 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:33.981747 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0528 04:37:33.981895 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:33.981856 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0528 04:37:33.981951 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:33.981995 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:33.982028 26677 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0528 04:37:33.982063 26677 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0528 04:37:33.982115 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:33.982149 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:33.982200 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:33.982369 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:33.982436 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:33.982484 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:33.982338 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0528 04:37:33.982515 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0528 04:37:33.982600 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:33.982635 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:33.982667 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:33.982701 26677 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0528 04:37:33.982756 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:33.982784 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0528 04:37:33.982874 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:33.982988 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0528 04:37:33.983026 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.983071 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.983146 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], Type: "network" } I0528 04:37:33.983071 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0528 04:37:33.983685 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.983818 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0528 04:37:33.983889 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0528 04:37:33.984252 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.984396 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0528 04:37:33.984445 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.984514 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:33.984589 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0528 04:37:33.984673 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.984737 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0528 04:37:33.984798 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0528 04:37:33.984897 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0528 04:37:33.985201 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0528 04:37:33.985370 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.985457 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:33.985535 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:33.985619 26677 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0528 04:37:33.985575 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0528 04:37:33.985692 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0528 04:37:33.985736 26677 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0528 04:37:33.985784 26677 api.go:53] PageSize not implemented I0528 04:37:33.985747 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0528 04:37:33.985849 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0528 04:37:33.986044 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0528 04:37:33.986093 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0528 04:37:33.986131 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0528 04:37:33.986229 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0528 04:37:33.986373 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0528 04:37:33.986521 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0528 04:37:33.986609 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0528 04:37:33.986660 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0528 04:37:33.986750 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0528 04:37:33.986895 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0528 04:37:33.987048 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0528 04:37:33.987188 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0528 04:37:33.987330 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0528 04:37:33.987466 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:33.987514 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0528 04:37:33.987669 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0528 04:37:33.987826 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0528 04:37:33.987927 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0528 04:37:33.988066 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:33.988106 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0528 04:37:33.988260 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:33.988304 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0528 04:37:33.988339 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0528 04:37:33.988442 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:33.988513 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:33.988595 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.988650 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.988664 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:33.988587 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.988841 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:33.988890 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:33.988906 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:33.988826 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:33.991827 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:33.991975 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.992053 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.992187 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.992250 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:33.993685 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:33.993836 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:33.993923 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0528 04:37:33.993953 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:33.993982 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.994018 26677 dns.go:232] Pre-creating DNS records I0528 04:37:33.994034 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:33.998570 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:34.000905 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.000963 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.038685 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.183371 26677 iamrole.go:89] ListRoles: { } I0528 04:37:34.184662 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:34.184864 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.185018 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:34.185045 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:34.185085 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.185115 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.185109 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0528 04:37:34.185218 26677 api.go:53] PageSize not implemented I0528 04:37:34.185229 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:34.185356 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.185498 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:34.185596 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0528 04:37:34.185735 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:34.185833 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:34.187139 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:34.187239 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.187294 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:34.187369 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:34.187376 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.187333 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.187413 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:34.187417 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:34.187477 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:34.187544 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.187571 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:34.187492 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:34.187650 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:34.187719 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.187855 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.188013 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:34.188063 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:34.188222 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:34.188281 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:34.188475 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:34.188521 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.188604 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.188676 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.188750 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.188849 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.188943 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.189131 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:34.189175 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:34.189224 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:34.189351 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.189423 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.189496 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.189515 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.189557 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.189602 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.189531 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.189644 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.189712 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.189723 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.189830 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.189834 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0528 04:37:34.189892 26677 api.go:53] PageSize not implemented I0528 04:37:34.189944 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.189995 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.190072 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.190206 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:34.190238 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.190275 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.190251 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:34.190297 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:34.190288 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.190365 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.190421 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.190437 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.190591 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:34.190623 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.193082 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:34.193146 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.193241 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.193302 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.193427 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.198679 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:34.198865 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:34.198951 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0528 04:37:34.202783 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.205068 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.211808 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:34.214080 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.214137 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.252534 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.378680 26677 iamrole.go:89] ListRoles: { } I0528 04:37:34.379929 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:34.380064 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.380148 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:34.380167 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:34.380204 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.380191 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:34.380228 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.380306 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.380387 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0528 04:37:34.380540 26677 api.go:53] PageSize not implemented I0528 04:37:34.380572 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:34.380709 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:34.380822 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:34.380931 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:34.382174 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:34.382263 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.382376 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:34.382388 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.382437 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.382456 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:34.382462 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:34.382492 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:34.382459 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.382510 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:34.382566 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.382590 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:34.382599 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:34.382632 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:34.382753 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:34.382798 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:34.382899 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:34.382941 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.383036 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:34.383062 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0528 04:37:34.383157 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:34.383183 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.383279 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.383375 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.383444 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.383497 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:34.383600 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:34.383696 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.383795 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:34.383908 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.383999 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.384025 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.384064 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.384076 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.384100 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.384120 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.384117 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.384143 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.384162 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.384180 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.384200 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.384286 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0528 04:37:34.384308 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.384307 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.384327 26677 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0528 04:37:34.384345 26677 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0528 04:37:34.384364 26677 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0528 04:37:34.384416 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0528 04:37:34.384453 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.384472 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } W0528 04:37:34.384495 26677 api.go:53] PageSize not implemented I0528 04:37:34.384491 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.384639 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.384729 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:34.384772 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:34.384772 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.384838 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.384818 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:34.384857 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.385036 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:34.385059 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.385095 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.385126 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.385135 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.391266 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:34.391433 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.391739 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.391930 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.392076 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.394677 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:34.395024 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:34.395184 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0528 04:37:34.395235 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.395270 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:34.395335 26677 dns.go:232] Pre-creating DNS records I0528 04:37:34.402973 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:34.407064 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.407158 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.446458 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.574985 26677 iamrole.go:89] ListRoles: { } I0528 04:37:34.576256 26677 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0528 04:37:34.576531 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:34.576658 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.576763 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.576768 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:34.576830 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:34.577803 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:34.577881 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0528 04:37:34.578388 26677 api.go:53] PageSize not implemented I0528 04:37:34.578356 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:34.578471 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:34.578544 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0528 04:37:34.578704 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:34.578806 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.582000 26677 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0528 04:37:34.582133 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.582215 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:34.582305 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:34.582240 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:34.582347 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:34.582402 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.582412 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:34.582469 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:34.582427 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:34.582505 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.582538 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:34.582590 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.582607 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:34.582614 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:34.582636 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:34.582757 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:34.582804 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.582963 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:34.583008 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.583102 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.583195 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.583336 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.583427 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:34.583555 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.583678 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:34.583825 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:34.583878 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.583999 26677 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0528 04:37:34.584144 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.584211 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.584255 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.584331 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.584348 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.584390 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.584378 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.584426 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.584464 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.584498 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.584499 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.584528 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0528 04:37:34.584611 26677 api.go:53] PageSize not implemented I0528 04:37:34.584579 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.584880 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.585085 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.585324 26677 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0528 04:37:34.585374 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.585427 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.585394 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:34.585463 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:34.585447 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.585515 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.585562 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.585572 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.585629 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:34.585655 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.588406 26677 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0528 04:37:34.588491 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.588614 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.588697 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.588800 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.590171 26677 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0528 04:37:34.590288 26677 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0528 04:37:34.590350 26677 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0528 04:37:34.592442 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.594455 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.601363 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:34.603680 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.603741 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.642718 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.824158 26677 iamrole.go:89] ListRoles: { } I0528 04:37:34.825498 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0528 04:37:34.825699 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0528 04:37:34.825824 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0528 04:37:34.825912 26677 api.go:53] PageSize not implemented I0528 04:37:34.825913 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:34.825970 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:34.826074 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:34.826140 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:34.826196 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.826389 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:34.826523 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:34.826567 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:34.826631 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.826686 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.826829 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:34.827594 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:34.827688 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0528 04:37:34.827725 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:34.827811 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.827847 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.827879 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:34.827885 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:34.827910 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:34.827864 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.827948 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:34.827998 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.828004 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:34.828018 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:34.828038 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:34.828164 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.828262 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.829012 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:34.829342 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.829545 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.829672 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:34.829837 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:34.829954 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:34.830119 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:34.830162 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:34.830251 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:34.830393 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:34.830441 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:34.830483 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:34.831042 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:34.831281 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.831601 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.831694 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.831714 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.831803 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.831891 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.832146 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.832194 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.832223 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.832240 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:34.832275 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:34.832305 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:34.832323 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0528 04:37:34.832396 26677 api.go:53] PageSize not implemented I0528 04:37:34.832507 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:34.832615 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:34.833054 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:34.833097 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.833148 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.833161 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.833106 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:34.833231 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:34.833251 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:34.833267 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.833296 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:34.833354 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:34.833368 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:34.836153 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:34.836230 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.836337 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.836392 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.836515 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.838381 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.838423 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.838524 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:34.838718 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:34.838815 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:34.838855 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:34.838891 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:34.838946 26677 dns.go:232] Pre-creating DNS records I0528 04:37:34.846207 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:34.849767 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.849859 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:34.888376 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.037874 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.040148 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0528 04:37:35.040348 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.040501 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:35.040533 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W0528 04:37:35.040550 26677 api.go:53] PageSize not implemented I0528 04:37:35.040573 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:35.040506 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:35.040659 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:35.040775 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.040825 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.040966 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:35.041044 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.041121 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:35.041183 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:35.041236 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.042432 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:35.042521 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.042585 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.042603 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.042633 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:35.042698 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.042684 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.042728 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.042722 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.042784 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:35.042820 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.042776 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:35.042839 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:35.042892 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.042937 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.043059 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:35.043087 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.043183 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:35.043210 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.043305 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.043390 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:35.043506 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.043555 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.043712 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:35.043753 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.043853 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:35.043984 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.044070 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.044196 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:35.044296 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.044362 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.044344 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.044447 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.044460 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.044454 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.044485 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.044581 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.044621 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.044645 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.044664 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0528 04:37:35.044730 26677 api.go:53] PageSize not implemented I0528 04:37:35.044720 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.044793 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.044876 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.044985 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.045258 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:35.045284 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.045322 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.045321 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:35.045350 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.045386 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:35.045402 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:35.045428 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.045467 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.045510 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.045518 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.048709 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:35.048792 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.048947 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.049071 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.049199 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.050645 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:35.050796 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:35.050877 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0528 04:37:35.053728 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.055638 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.055685 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.062044 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:35.064456 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.064518 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.064554 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.102097 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.268692 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.270024 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0528 04:37:35.270277 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:35.270402 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.270567 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:35.270641 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0528 04:37:35.270693 26677 api.go:53] PageSize not implemented I0528 04:37:35.270689 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.271749 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.271871 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.271951 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:35.271972 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:35.272024 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.272312 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:35.272448 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:35.272579 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:35.276377 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:35.276504 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.276506 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.276580 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:35.276673 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.276722 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:35.276790 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.276634 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.276833 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:35.276896 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.276930 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.276986 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.277102 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:35.277256 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.277308 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.277381 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.277504 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:35.277530 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:35.277630 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.277665 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.277722 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.277803 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:35.277823 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.277869 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.277959 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:35.277980 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:35.278088 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.278158 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.278365 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:35.278577 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.278654 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.278590 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.278668 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.278698 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.278727 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.278752 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.278781 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.278748 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.278798 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.278861 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0528 04:37:35.278905 26677 api.go:53] PageSize not implemented I0528 04:37:35.279163 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.279371 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.279556 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.279827 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.280026 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:35.280109 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.280220 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.280139 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:35.280280 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:35.280266 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.280490 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:35.280571 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.280663 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.280737 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.280775 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.285066 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:35.285207 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.285346 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.285472 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.285644 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.287742 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:35.288027 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:35.288169 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:35.288224 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.288306 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:35.288369 26677 dns.go:232] Pre-creating DNS records I0528 04:37:35.294330 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:35.297202 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.297278 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.297317 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.335787 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.498023 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.499325 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0528 04:37:35.499755 26677 api.go:53] PageSize not implemented I0528 04:37:35.499663 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.499871 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.499952 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:35.499910 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:35.499973 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:35.499995 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:35.500100 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:35.500159 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.500253 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:35.500310 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.500401 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:35.500457 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:35.500508 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.501291 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:35.501385 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.501387 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.501436 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:35.501454 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.501498 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.501474 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.501524 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:35.501569 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:35.501512 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:35.501623 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.501642 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.501695 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.501740 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.501849 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:35.501875 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.501952 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:35.501975 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.502027 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:35.502110 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.502142 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.502192 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.502263 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:35.502289 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.502367 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.502446 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:35.502538 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.502584 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:35.502727 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:35.502841 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.502928 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.502914 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.503017 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.502993 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.503035 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.503079 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.503111 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.503117 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.503145 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.503178 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.503207 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.503221 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0528 04:37:35.503287 26677 api.go:53] PageSize not implemented I0528 04:37:35.503276 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.503426 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.503644 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:35.503684 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:35.503728 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:35.503738 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.503781 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.503797 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.503907 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.503956 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.503971 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.504077 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:35.504103 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.509083 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:35.509220 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.509408 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.509550 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.510119 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.511816 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:35.511968 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:35.512064 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0528 04:37:35.513891 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.515872 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.515923 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.516003 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.521776 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:35.524300 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.524357 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.524392 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.524421 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.562210 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.696152 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.698236 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0528 04:37:35.698419 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:35.698543 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0528 04:37:35.698652 26677 api.go:53] PageSize not implemented I0528 04:37:35.698635 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.698702 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:35.698811 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.698945 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0528 04:37:35.699055 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.699184 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:35.699286 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:35.699384 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.699479 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:35.699507 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:35.699559 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.700701 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:35.700805 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.700871 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.700839 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:35.700925 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:35.700963 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:35.701020 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.701047 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:35.701049 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.701069 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:35.701112 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.701130 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.701169 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.701232 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.701322 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:35.701345 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.701428 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:35.701452 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.701529 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0528 04:37:35.701616 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.701689 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.701732 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:35.701820 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.701845 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.701892 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.701934 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.701979 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.702049 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:35.702093 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:35.702227 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.702294 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.702310 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.702347 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.702378 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.702246 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.702412 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.702443 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.702471 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0528 04:37:35.702549 26677 api.go:53] PageSize not implemented I0528 04:37:35.702579 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.702659 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.702761 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.702824 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.702868 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.702993 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.703090 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:35.703110 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.703141 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.703153 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.703161 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:35.703242 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:35.703236 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.703289 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.703309 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.703384 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:35.703410 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.705962 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:35.706028 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.706124 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.706178 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.706320 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.708737 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:35.709000 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:35.709090 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:35.709126 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.709155 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:35.709203 26677 dns.go:232] Pre-creating DNS records I0528 04:37:35.716343 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0528 04:37:35.719980 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.720062 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.720115 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.720158 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.759268 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.880194 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.881522 26677 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0528 04:37:35.881775 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0528 04:37:35.881873 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0528 04:37:35.881926 26677 api.go:53] PageSize not implemented I0528 04:37:35.881965 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:35.881967 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.882060 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0528 04:37:35.882143 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.882344 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0528 04:37:35.882464 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0528 04:37:35.882597 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0528 04:37:35.882699 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.882793 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:35.882825 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:35.882881 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.885040 26677 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0528 04:37:35.885241 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.885660 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.885817 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0528 04:37:35.885914 26677 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.885939 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.886050 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0528 04:37:35.885899 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.885963 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.886220 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0528 04:37:35.886220 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0528 04:37:35.886618 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.887135 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.886663 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.887219 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:35.887272 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:35.887296 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.887359 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.887414 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.887501 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0528 04:37:35.887624 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.888176 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:35.888254 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0528 04:37:35.888337 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0528 04:37:35.888362 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.888428 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0528 04:37:35.888501 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.888531 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.888625 26677 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0528 04:37:35.888748 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.888823 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.888856 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.888908 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.888920 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.888950 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.888971 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0528 04:37:35.888995 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0528 04:37:35.889014 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0528 04:37:35.889031 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0528 04:37:35.889093 26677 api.go:53] PageSize not implemented I0528 04:37:35.889106 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.889227 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.889338 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.889450 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:35.889787 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:35.890113 26677 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0528 04:37:35.890187 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.890270 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.890217 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:35.890299 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.890343 26677 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0528 04:37:35.890392 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0528 04:37:35.890441 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.890516 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.890585 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.890601 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.893456 26677 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0528 04:37:35.893590 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.893748 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.893873 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.894103 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.897351 26677 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0528 04:37:35.897583 26677 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0528 04:37:35.897689 26677 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0528 04:37:35.897810 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:35.897844 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:35.897867 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:35.897885 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:35.897904 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:35.897922 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:35.897941 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:35.897961 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:35.897981 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:35.898005 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:35.898026 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:35.898044 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:35.898065 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:35.898084 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:35.898106 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:35.898127 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:35.898148 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:35.898168 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:35.898194 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:35.898212 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:35.898233 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0528 04:37:35.898265 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:35.898294 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:35.898323 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:35.898351 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:35.898373 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:35.898394 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:35.899988 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:35.900028 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:35.900041 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:35.900063 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900099 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0528 04:37:35.900138 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900178 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900208 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900242 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900268 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900296 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0528 04:37:35.900329 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900375 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0528 04:37:35.900400 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0528 04:37:35.900432 26677 api.go:53] PageSize not implemented I0528 04:37:35.900445 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:35.900459 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:35.900468 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0528 04:37:35.900492 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:35.900506 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:35.900517 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0528 04:37:35.900561 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:35.900587 26677 iamrole.go:89] ListRoles: { } I0528 04:37:35.900618 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:35.900636 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:35.900663 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:35.900692 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:35.900720 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0528 04:37:35.901068 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:nodes.complex.example.com ok I0528 04:37:35.901124 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0528 04:37:35.901174 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0528 04:37:35.901141 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I0528 04:37:35.901205 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok subnet:subnet-3 ok I0528 04:37:35.901170 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0528 04:37:35.901232 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok subnet:subnet-2 ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok autoscaling-config:lt-1 ok I0528 04:37:35.901278 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:35.901227 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0528 04:37:35.901354 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0528 04:37:35.901275 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } iam-instance-profile:masters.complex.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0528 04:37:35.901230 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0528 04:37:35.901318 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0528 04:37:35.901509 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:35.901529 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0528 04:37:35.901596 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:35.901674 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0528 04:37:35.901710 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:35.901768 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0528 04:37:35.901806 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:35.901862 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:35.901909 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 } ] } W0528 04:37:35.902258 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:35.902275 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:35.902358 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0528 04:37:35.902371 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0528 04:37:35.902416 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:35.902440 26677 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0528 04:37:35.902468 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0528 04:37:35.902502 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0528 04:37:35.902548 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0528 04:37:35.902601 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0528 04:37:35.902617 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0528 04:37:35.902646 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0528 04:37:35.902707 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:35.902783 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:35.902861 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:35.903166 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.17s) === RUN TestLifecycleExternalLB I0528 04:37:35.903354 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:35.903499 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:35.903538 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:35.903578 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:35.903616 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:35.903662 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:35.903705 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:35.903734 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:35.903773 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:35.903801 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:35.903853 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0528 04:37:35.929423 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:35.932144 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.932229 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:35.970609 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.040318 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.041233 26677 executor.go:111] Tasks: 0 done / 93 total; 49 can run W0528 04:37:36.041368 26677 api.go:53] PageSize not implemented I0528 04:37:36.041487 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.041636 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0528 04:37:36.041635 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.041737 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0528 04:37:36.041763 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0528 04:37:36.041756 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.041832 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0528 04:37:36.041871 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0528 04:37:36.041889 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } I0528 04:37:36.041849 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0528 04:37:36.041860 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0528 04:37:36.041951 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } W0528 04:37:36.041916 26677 api.go:53] PageSize not implemented I0528 04:37:36.042006 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:36.042031 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } W0528 04:37:36.042040 26677 api.go:53] PageSize not implemented I0528 04:37:36.041983 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] } I0528 04:37:36.042008 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.043135 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.043429 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0528 04:37:36.043518 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I0528 04:37:36.043631 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.043771 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] }] } I0528 04:37:36.043895 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.043917 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.044057 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.044136 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.044331 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.044410 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:36.044453 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:36.044493 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:36.044551 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.044591 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:36.045376 26677 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0528 04:37:36.045719 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0528 04:37:36.046055 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0528 04:37:36.045815 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0528 04:37:36.046200 26677 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0528 04:37:36.046341 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0528 04:37:36.046504 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.046817 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:36.046848 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.046912 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.046971 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0528 04:37:36.047035 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.047084 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.047505 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:36.047091 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0528 04:37:36.047563 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.047080 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0528 04:37:36.047593 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.047624 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.047732 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:36.047812 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:36.047864 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:36.047903 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:36.048003 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.048057 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.048099 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:36.048150 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:36.048293 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:36.048379 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:36.048421 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] } I0528 04:37:36.048336 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0528 04:37:36.049992 26677 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0528 04:37:36.050142 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.050296 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.050390 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:36.050521 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.050667 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.050812 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.050975 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.051119 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.051278 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.051352 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.051430 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.051499 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.051575 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.051688 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.051838 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" }] }] } I0528 04:37:36.051962 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.052026 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.052135 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" }] }] } I0528 04:37:36.052317 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:36.052376 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:36.052415 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0528 04:37:36.052548 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.052798 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0528 04:37:36.052936 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:36.052962 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:36.053701 26677 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0528 04:37:36.053861 26677 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0528 04:37:36.053983 26677 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0528 04:37:36.054014 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.054045 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.054081 26677 dns.go:232] Pre-creating DNS records I0528 04:37:36.054101 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.057515 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.059252 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.059312 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.096272 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.167876 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.169070 26677 executor.go:111] Tasks: 0 done / 93 total; 49 can run W0528 04:37:36.169339 26677 api.go:53] PageSize not implemented W0528 04:37:36.169442 26677 api.go:53] PageSize not implemented I0528 04:37:36.169373 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0528 04:37:36.169516 26677 api.go:53] PageSize not implemented I0528 04:37:36.169544 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:36.169587 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:36.169531 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.169642 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.169673 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:36.169875 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.169867 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:36.169959 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0528 04:37:36.169932 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.170038 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0528 04:37:36.170087 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0528 04:37:36.170104 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } I0528 04:37:36.170072 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.170145 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0528 04:37:36.170181 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0528 04:37:36.170196 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0528 04:37:36.170232 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.172120 26677 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0528 04:37:36.172256 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0528 04:37:36.172243 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.172486 26677 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0528 04:37:36.172393 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0528 04:37:36.172596 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.172656 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.173753 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.173874 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.173983 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:36.174019 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:36.174098 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0528 04:37:36.174346 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.179774 26677 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0528 04:37:36.179909 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.180055 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.180178 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.180341 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.180645 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.180890 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.181113 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.183300 26677 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0528 04:37:36.183542 26677 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0528 04:37:36.183664 26677 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0528 04:37:36.186928 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:36.186988 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:36.187028 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:36.187064 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:36.187101 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:36.187135 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:36.187171 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:36.187209 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:36.187247 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:36.187288 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:36.187332 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:36.187365 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:36.187399 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:36.187435 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:36.187469 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.187506 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.187542 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:36.187582 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:36.187617 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:36.187653 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:36.187690 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.187728 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.187768 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:36.187805 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.189432 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:36.189489 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:36.189507 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:36.189546 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189593 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0528 04:37:36.189647 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189708 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189756 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189811 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189855 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189896 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0528 04:37:36.189943 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.189995 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0528 04:37:36.190036 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0528 04:37:36.190088 26677 api.go:53] PageSize not implemented I0528 04:37:36.190108 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:36.190136 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.190155 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:36.190179 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:36.190198 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:36.190247 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:36.190290 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.190348 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.190391 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.190445 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:36.190503 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:36.190552 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-1 ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0528 04:37:36.191328 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.externallb.example.com ok keypair:key-1 ok I0528 04:37:36.191380 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0528 04:37:36.191417 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0528 04:37:36.191474 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0528 04:37:36.191275 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0528 04:37:36.191509 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.191554 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0528 04:37:36.191572 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0528 04:37:36.191706 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.191727 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:36.191672 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0528 04:37:36.191764 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.191826 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0528 04:37:36.191954 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.191973 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:36.192004 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:36.192056 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0528 04:37:36.192096 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:36.192162 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:36.192179 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0528 04:37:36.192238 26677 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0528 04:37:36.192257 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0528 04:37:36.192292 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0528 04:37:36.192330 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0528 04:37:36.192423 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0528 04:37:36.192442 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0528 04:37:36.192478 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0528 04:37:36.192556 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:36.192693 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:36.192832 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:36.193217 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.29s) === RUN TestLifecycleSharedSubnet I0528 04:37:36.193401 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:36.193540 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:36.193578 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:36.193619 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:36.193658 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:36.193708 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:36.193750 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:36.193780 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:36.193817 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:36.193844 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:36.193894 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0528 04:37:36.214004 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.216058 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.216123 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.253522 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.335035 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.335814 26677 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0528 04:37:36.336017 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.336272 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0528 04:37:36.336253 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.336371 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.336413 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.336381 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I0528 04:37:36.336474 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.336558 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0528 04:37:36.336635 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.336658 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0528 04:37:36.336730 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.336777 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:36.336841 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:36.336862 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:36.336895 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.337020 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.338014 26677 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0528 04:37:36.338195 26677 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0528 04:37:36.338153 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0528 04:37:36.338266 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0528 04:37:36.338293 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0528 04:37:36.338328 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0528 04:37:36.338390 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0528 04:37:36.338502 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.338522 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0528 04:37:36.338564 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.338621 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.338660 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.339737 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0528 04:37:36.340277 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.340375 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.340444 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.340494 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:36.340556 26677 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0528 04:37:36.341002 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0528 04:37:36.343627 26677 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0528 04:37:36.343778 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.343958 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0528 04:37:36.344038 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.344095 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.344209 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.344373 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.344505 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.344630 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0528 04:37:36.344753 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.344872 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.344985 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.345100 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.345264 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:36.345399 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.345535 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0528 04:37:36.345679 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0528 04:37:36.345800 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.345847 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.347243 26677 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0528 04:37:36.347434 26677 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0528 04:37:36.347580 26677 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0528 04:37:36.347613 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.347636 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.347675 26677 dns.go:232] Pre-creating DNS records I0528 04:37:36.347695 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.351081 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.352914 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.352968 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.389953 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.458890 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.459709 26677 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0528 04:37:36.459891 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.459921 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.460054 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:36.460151 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:36.460176 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:36.460230 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.460325 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.461983 26677 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0528 04:37:36.462078 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0528 04:37:36.462239 26677 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0528 04:37:36.462207 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0528 04:37:36.462284 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0528 04:37:36.462340 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.462378 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0528 04:37:36.462532 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.462587 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.465109 26677 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0528 04:37:36.465189 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.465293 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.465457 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.465581 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.466981 26677 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0528 04:37:36.467131 26677 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0528 04:37:36.467225 26677 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0528 04:37:36.468854 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:36.468891 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:36.468912 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:36.468930 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:36.468949 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:36.468968 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:36.468990 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:36.469012 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:36.469036 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:36.469054 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:36.469071 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:36.469089 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.469109 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.469128 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:36.469149 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:36.469169 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:36.469187 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.469208 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.469228 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.470035 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:36.470073 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:36.470083 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:36.470104 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470133 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0528 04:37:36.470164 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470198 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470221 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470244 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470273 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470295 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0528 04:37:36.470316 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470341 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0528 04:37:36.470360 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0528 04:37:36.470388 26677 api.go:53] PageSize not implemented I0528 04:37:36.470401 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:36.470416 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.470428 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:36.470442 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:36.470453 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:36.470484 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:36.470509 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.470537 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.470555 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.470583 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:36.470605 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:36.470819 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0528 04:37:36.470859 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.sharedsubnet.example.com ok I0528 04:37:36.470919 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0528 04:37:36.470968 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0528 04:37:36.470879 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I0528 04:37:36.470995 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0528 04:37:36.471050 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0528 04:37:36.470913 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0528 04:37:36.471087 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.471100 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0528 04:37:36.471134 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.471195 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0528 04:37:36.471278 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.471298 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:36.471330 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:36.471370 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0528 04:37:36.471449 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0528 04:37:36.471495 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0528 04:37:36.471509 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0528 04:37:36.471530 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0528 04:37:36.471549 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0528 04:37:36.471599 26677 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0528 04:37:36.471607 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0528 04:37:36.471625 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0528 04:37:36.471780 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.28s) === RUN TestLifecyclePrivateSharedSubnet I0528 04:37:36.471922 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:36.472019 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:36.472042 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:36.472068 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:36.472092 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:36.472121 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:36.472144 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:36.472161 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:36.472190 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:36.472207 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:36.472237 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0528 04:37:36.497997 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.501184 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.501265 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.501326 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.538679 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.626522 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.628496 26677 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0528 04:37:36.628690 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.628762 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:36.628885 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:36.628932 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:36.628987 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.629075 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.629159 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] } I0528 04:37:36.629166 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.629273 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0528 04:37:36.629355 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.629386 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0528 04:37:36.629430 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.629476 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.629537 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0528 04:37:36.629612 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.629928 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.630117 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.630252 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.631125 26677 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0528 04:37:36.631290 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0528 04:37:36.631297 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.631348 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0528 04:37:36.631430 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0528 04:37:36.631391 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0528 04:37:36.631514 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:36.631516 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.631568 26677 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I0528 04:37:36.631513 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-su-5ol32q", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-su-5ol32q" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0528 04:37:36.631660 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0528 04:37:36.631709 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0528 04:37:36.631790 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.631856 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.631887 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.631930 26677 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0528 04:37:36.631914 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.631993 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0528 04:37:36.632064 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0528 04:37:36.632029 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.632132 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.632106 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0528 04:37:36.632180 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.632181 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.632223 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0528 04:37:36.632422 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0528 04:37:36.632558 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.632613 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0528 04:37:36.632665 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.632714 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:36.632740 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:36.632761 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:36.632788 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:36.632821 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:36.632889 26677 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0528 04:37:36.634177 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.634766 26677 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0528 04:37:36.634927 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0528 04:37:36.635030 26677 api.go:53] PageSize not implemented I0528 04:37:36.635051 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:36.635104 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.635181 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.635135 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.635241 26677 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0528 04:37:36.635282 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.635172 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-su-5ol32q", SubnetMappings: [{ SubnetId: "subnet-abcdef" }], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }], Type: "network" } I0528 04:37:36.635301 26677 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0528 04:37:36.635336 26677 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0528 04:37:36.635299 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0528 04:37:36.635482 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.635618 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.635742 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:36.635831 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.636037 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:36.636143 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.636274 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" }] }] } I0528 04:37:36.636439 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0528 04:37:36.636480 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.636588 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1", Port: 22, Protocol: "TCP" } I0528 04:37:36.636610 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0528 04:37:36.636743 26677 api.go:53] PageSize not implemented I0528 04:37:36.636768 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.636753 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.636896 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.637058 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.637172 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.637269 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.637410 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0528 04:37:36.637557 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.637699 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0528 04:37:36.637826 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.637978 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0528 04:37:36.638114 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.638263 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.638400 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.638534 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0528 04:37:36.638702 26677 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0528 04:37:36.638739 26677 api.go:53] PageSize not implemented I0528 04:37:36.638798 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.638826 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0528 04:37:36.638936 26677 api.go:53] PageSize not implemented I0528 04:37:36.638975 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.638982 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:36.641969 26677 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0528 04:37:36.642081 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.642156 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.642323 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.642442 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.644139 26677 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0528 04:37:36.644371 26677 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0528 04:37:36.644525 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:36.644583 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.644619 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.644689 26677 dns.go:232] Pre-creating DNS records I0528 04:37:36.644715 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.651232 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.654531 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.654617 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.654682 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.693809 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.771938 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.773146 26677 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0528 04:37:36.773465 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.773832 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.773807 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.773955 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:36.774093 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:36.774129 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:36.774202 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.775587 26677 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0528 04:37:36.775834 26677 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0528 04:37:36.775869 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0528 04:37:36.775903 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.775986 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0528 04:37:36.776016 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0528 04:37:36.775957 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0528 04:37:36.776192 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0528 04:37:36.776250 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0528 04:37:36.776138 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0528 04:37:36.776373 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.776432 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0528 04:37:36.776630 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:36.776686 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0528 04:37:36.776866 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:36.776943 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0528 04:37:36.777058 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0528 04:37:36.777194 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.778030 26677 executor.go:111] Tasks: 67 done / 107 total; 27 can run W0528 04:37:36.778300 26677 api.go:53] PageSize not implemented I0528 04:37:36.778323 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.778380 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.778404 26677 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0528 04:37:36.778377 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.778394 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:36.778499 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.778501 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.778521 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0528 04:37:36.778584 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0528 04:37:36.778620 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } I0528 04:37:36.778657 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0528 04:37:36.778691 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } W0528 04:37:36.778769 26677 api.go:53] PageSize not implemented I0528 04:37:36.778790 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.778865 26677 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0528 04:37:36.778900 26677 api.go:53] PageSize not implemented I0528 04:37:36.778941 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.779032 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0528 04:37:36.779118 26677 api.go:53] PageSize not implemented I0528 04:37:36.779130 26677 dnsname.go:118] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0528 04:37:36.779154 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.779309 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.783600 26677 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0528 04:37:36.783704 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.783869 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.784020 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.784180 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.786199 26677 executor.go:111] Tasks: 103 done / 107 total; 2 can run W0528 04:37:36.786372 26677 api.go:53] PageSize not implemented I0528 04:37:36.786403 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.786613 26677 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0528 04:37:36.786787 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:36.790087 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:36.790153 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:36.790192 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:36.790224 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:36.790259 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:36.790292 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:36.790329 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:36.790365 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:36.790407 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0528 04:37:36.790443 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:36.790473 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:36.790504 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:36.790535 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.790569 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.790603 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:36.790636 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:36.790670 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:36.790709 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:36.790743 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:36.790775 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.790811 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.790846 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.792380 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:36.792448 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:36.792468 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:36.792507 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792565 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0528 04:37:36.792631 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792694 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792734 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792775 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792822 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792856 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0528 04:37:36.792893 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792936 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0528 04:37:36.792976 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0528 04:37:36.793022 26677 api.go:53] PageSize not implemented I0528 04:37:36.793038 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0528 04:37:36.793076 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:36.793100 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.793113 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0528 04:37:36.793150 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:36.793172 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:36.793187 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0528 04:37:36.793246 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:36.793290 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.793337 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.793362 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.793404 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:36.793444 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:36.793946 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.794014 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0528 04:37:36.793986 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.private-shared-subnet.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0528 04:37:36.794086 26677 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0528 04:37:36.794070 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.794158 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0528 04:37:36.794156 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1 ok I0528 04:37:36.794078 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 } ] } I0528 04:37:36.794217 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } W0528 04:37:36.794241 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.794253 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0528 04:37:36.794142 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok I0528 04:37:36.794263 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:36.794278 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4 ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0528 04:37:36.794293 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:36.794349 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0528 04:37:36.794549 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.794571 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0528 04:37:36.794613 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.794660 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0528 04:37:36.794767 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.794787 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:36.794819 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0528 04:37:36.794904 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0528 04:37:36.795005 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:36.795088 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.795141 26677 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0528 04:37:36.795160 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0528 04:37:36.795196 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0528 04:37:36.795105 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:36.795229 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.795271 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0528 04:37:36.795289 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0528 04:37:36.795318 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0528 04:37:36.795244 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0528 04:37:36.795356 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:36.795373 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0528 04:37:36.795352 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0528 04:37:36.795420 26677 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0528 04:37:36.795435 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0528 04:37:36.795471 26677 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0528 04:37:36.795706 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.32s) === RUN TestLifecyclePrivateSharedIP I0528 04:37:36.795888 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:36.796021 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:36.796055 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:36.796094 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:36.796132 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:36.796180 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:36.796218 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:36.796248 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:36.796285 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:36.796311 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:36.796363 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0528 04:37:36.825111 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:36.828464 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.828543 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.828601 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.867698 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.965313 26677 iamrole.go:89] ListRoles: { } I0528 04:37:36.966715 26677 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0528 04:37:36.966843 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:36.967008 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:36.968382 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:36.968427 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:36.967048 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0528 04:37:36.968542 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0528 04:37:36.968672 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:36.968853 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.968980 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:36.969016 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:36.969144 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:36.969251 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:36.969283 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:36.969411 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.968696 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0528 04:37:36.969621 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0528 04:37:36.969707 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0528 04:37:36.969804 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0528 04:37:36.972950 26677 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0528 04:37:36.973038 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0528 04:37:36.973090 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:36.973825 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0528 04:37:36.973905 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0528 04:37:36.973957 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0528 04:37:36.973984 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0528 04:37:36.974041 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0528 04:37:36.973243 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I0528 04:37:36.974064 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0528 04:37:36.974108 26677 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-ip-eepmph", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-ip-eepmph" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0528 04:37:36.974135 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0528 04:37:36.974320 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0528 04:37:36.974323 26677 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0528 04:37:36.974437 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0528 04:37:36.974488 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0528 04:37:36.974414 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.974534 26677 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0528 04:37:36.974553 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:36.974616 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:36.974664 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0528 04:37:36.974793 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0528 04:37:36.974871 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:36.974956 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0528 04:37:36.975061 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:36.975111 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0528 04:37:36.975216 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:36.975349 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:36.975451 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.975548 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:36.975586 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:36.975515 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0528 04:37:36.975611 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:36.975728 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I0528 04:37:36.975760 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0528 04:37:36.975993 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.976140 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0528 04:37:36.976251 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:36.976301 26677 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] } I0528 04:37:36.976374 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:36.976431 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:36.976470 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:36.976523 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:36.976574 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:36.976625 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:36.976669 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:36.976710 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:36.976354 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0528 04:37:36.976843 26677 executor.go:111] Tasks: 69 done / 116 total; 32 can run W0528 04:37:36.977095 26677 api.go:53] PageSize not implemented I0528 04:37:36.977032 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:36.977168 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:36.977197 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:36.977254 26677 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0528 04:37:36.977206 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0528 04:37:36.977253 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } W0528 04:37:36.977338 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:36.977295 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:36.977443 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:36.977488 26677 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-ip-eepmph", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], Type: "network" } I0528 04:37:36.977470 26677 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0528 04:37:36.977611 26677 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0528 04:37:36.977581 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.977723 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" }] }] } I0528 04:37:36.977856 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0528 04:37:36.977987 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:36.978118 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0528 04:37:36.978168 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.978356 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0528 04:37:36.978443 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.978582 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.978769 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.978893 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.979025 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:36.979124 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.979197 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:36.979222 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:36.979247 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:36.979331 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:36.979290 26677 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1", Port: 22, Protocol: "TCP" } W0528 04:37:36.979419 26677 api.go:53] PageSize not implemented I0528 04:37:36.979444 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:36.979374 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.979583 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.979752 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.979887 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0528 04:37:36.979990 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.980206 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.980377 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0528 04:37:36.980424 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0528 04:37:36.980465 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.980569 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.980651 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.980853 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0528 04:37:36.980958 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0528 04:37:36.981114 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0528 04:37:36.981266 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.981403 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0528 04:37:36.981447 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0528 04:37:36.981598 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0528 04:37:36.981707 26677 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0528 04:37:36.981779 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I0528 04:37:36.981986 26677 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0528 04:37:36.982134 26677 api.go:53] PageSize not implemented I0528 04:37:36.982134 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:36.982182 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:36.982129 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.982405 26677 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:36.982366 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } W0528 04:37:36.982491 26677 api.go:53] PageSize not implemented I0528 04:37:36.982516 26677 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0528 04:37:36.982583 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:36.982614 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:36.982638 26677 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0528 04:37:36.982587 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:36.989332 26677 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0528 04:37:36.989439 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:36.989653 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0528 04:37:36.989824 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.989934 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.990130 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.990274 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:36.992780 26677 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0528 04:37:36.993103 26677 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0528 04:37:36.993259 26677 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0528 04:37:36.993309 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:36.993362 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.993419 26677 dns.go:232] Pre-creating DNS records I0528 04:37:36.993444 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:36.999624 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:37.003013 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.003101 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.003164 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.041401 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:37.142782 26677 iamrole.go:89] ListRoles: { } I0528 04:37:37.144179 26677 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0528 04:37:37.144327 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.145506 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:37.145652 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0528 04:37:37.145712 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0528 04:37:37.145738 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0528 04:37:37.145770 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.146026 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:37.149191 26677 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0528 04:37:37.149567 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0528 04:37:37.149615 26677 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0528 04:37:37.149645 26677 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0528 04:37:37.149613 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0528 04:37:37.149807 26677 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0528 04:37:37.149713 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0528 04:37:37.149853 26677 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0528 04:37:37.149767 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0528 04:37:37.150051 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:37.150133 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:37.150198 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.150740 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0528 04:37:37.150990 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] }] } I0528 04:37:37.151182 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:37.151234 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0528 04:37:37.151381 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.151491 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0528 04:37:37.151647 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.151749 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:37.151818 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:37.151857 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0528 04:37:37.151939 26677 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0528 04:37:37.152017 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:37.152118 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W0528 04:37:37.152207 26677 api.go:53] PageSize not implemented I0528 04:37:37.152199 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:37.152231 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.152215 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:37.152313 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:37.152286 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:37.152345 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0528 04:37:37.152307 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.152403 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0528 04:37:37.152432 26677 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0528 04:37:37.152454 26677 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0528 04:37:37.152512 26677 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0528 04:37:37.152564 26677 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } W0528 04:37:37.152664 26677 api.go:53] PageSize not implemented I0528 04:37:37.152692 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.152634 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.152834 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.153339 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.153495 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0528 04:37:37.153660 26677 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0528 04:37:37.153720 26677 api.go:53] PageSize not implemented I0528 04:37:37.153709 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.153777 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:37.153799 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.153788 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0528 04:37:37.154009 26677 dnsname.go:118] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" W0528 04:37:37.154044 26677 api.go:53] PageSize not implemented I0528 04:37:37.154076 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.154152 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.156923 26677 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0528 04:37:37.157013 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.157175 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.157352 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.157449 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.157629 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.159500 26677 executor.go:111] Tasks: 112 done / 116 total; 2 can run W0528 04:37:37.159606 26677 api.go:53] PageSize not implemented I0528 04:37:37.159633 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.159857 26677 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0528 04:37:37.159980 26677 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0528 04:37:37.163002 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:37.163057 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:37.163090 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:37.163127 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:37.163158 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:37.163189 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:37.163221 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:37.163257 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:37.163291 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0528 04:37:37.163362 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0528 04:37:37.163394 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:37.163424 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:37.163453 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:37.163485 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:37.163514 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0528 04:37:37.163545 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:37.163577 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:37.163610 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0528 04:37:37.163642 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0528 04:37:37.163673 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:37.163706 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:37.163736 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0528 04:37:37.163772 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:37.163804 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:37.163835 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:37.163868 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:37.163904 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:37.165295 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:37.165358 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:37.165375 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:37.165411 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165465 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0528 04:37:37.165526 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165582 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165618 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165657 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165697 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165738 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0528 04:37:37.165787 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165836 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:37.165861 26677 natgateway.go:117] DescribeNatGateways: { } I0528 04:37:37.165901 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0528 04:37:37.165934 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0528 04:37:37.165974 26677 api.go:53] PageSize not implemented I0528 04:37:37.165988 26677 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0528 04:37:37.166023 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:37.166045 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:37.166057 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0528 04:37:37.166092 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:37.166113 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:37.166127 26677 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0528 04:37:37.166187 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:37.166225 26677 iamrole.go:89] ListRoles: { } I0528 04:37:37.166269 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:37.166296 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:37.166336 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:37.166382 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:37.166414 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0528 04:37:37.166450 26677 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0528 04:37:37.166481 26677 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0528 04:37:37.167038 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0528 04:37:37.167067 26677 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0528 04:37:37.167114 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0528 04:37:37.167109 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0528 04:37:37.167142 26677 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0528 04:37:37.167202 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0528 04:37:37.167190 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4 ok iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-group:nodes.private-shared-ip.example.com ok I0528 04:37:37.167233 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } autoscaling-group:bastion.private-shared-ip.example.com ok keypair:key-1 ok I0528 04:37:37.166996 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:37.167253 26677 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok I0528 04:37:37.167285 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1 ok iam-instance-profile:masters.private-shared-ip.example.com ok I0528 04:37:37.167311 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0528 04:37:37.167354 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0528 04:37:37.167239 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0528 04:37:37.167449 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:37.167467 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0528 04:37:37.167496 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:37.167530 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 } ] } W0528 04:37:37.167648 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:37.167661 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I0528 04:37:37.167694 26677 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0528 04:37:37.167731 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0528 04:37:37.167771 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0528 04:37:37.167798 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0528 04:37:37.167866 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:37.167874 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0528 04:37:37.167925 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0528 04:37:37.167962 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0528 04:37:37.168023 26677 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0528 04:37:37.167980 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:37.168048 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0528 04:37:37.168073 26677 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0528 04:37:37.168101 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0528 04:37:37.168127 26677 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0528 04:37:37.168137 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0528 04:37:37.168160 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0528 04:37:37.168180 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0528 04:37:37.168210 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0528 04:37:37.168219 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0528 04:37:37.168092 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0528 04:37:37.168278 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:37.168302 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:37.168237 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0528 04:37:37.168333 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:37.168392 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:37.168425 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0528 04:37:37.168595 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.37s) === RUN TestLifecycleManyAddons I0528 04:37:37.168713 26677 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:37.168805 26677 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:37.168829 26677 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:37.168853 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:37.168877 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:37.168905 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:37.168931 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:37.168947 26677 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:37.168970 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:37.168992 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:37.169022 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0528 04:37:37.185070 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:37.187157 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.187223 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.225451 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:37.915745 26677 iamrole.go:89] ListRoles: { } I0528 04:37:37.916813 26677 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0528 04:37:37.917971 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:37.917093 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.918278 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:37.918701 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:37.918749 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.918903 26677 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0528 04:37:37.919105 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:37.920373 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:37.920486 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0528 04:37:37.920609 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0528 04:37:37.920886 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.922154 26677 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0528 04:37:37.921106 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.922246 26677 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0528 04:37:37.922886 26677 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.922955 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0528 04:37:37.922977 26677 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0528 04:37:37.922995 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:37.923022 26677 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.923086 26677 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.923182 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:37.923207 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:37.924840 26677 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0528 04:37:37.925098 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:37.925115 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0528 04:37:37.925236 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:37.925269 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:37.925342 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:37.925416 26677 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:37.925419 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:37.925524 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:37.925564 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:37.925594 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.925665 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0528 04:37:37.925633 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeTags\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:37.925741 26677 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:37.925811 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.926066 26677 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.926171 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:37.926228 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0528 04:37:37.926460 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:37.926667 26677 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:37.926733 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:37.926840 26677 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0528 04:37:37.926885 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:37.926924 26677 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0528 04:37:37.927013 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:37.927067 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:37.927030 26677 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" } ] }], VpcId: "vpc-1" } I0528 04:37:37.927113 26677 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I0528 04:37:37.927236 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:37.927292 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:37.928086 26677 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"acm:DescribeCertificate\",\n \"acm:ListCertificates\",\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateTags\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeInternetGateways\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSnapshots\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcPeeringConnections\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListenerCertificates\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeRules\",\n \"elasticloadbalancing:DescribeTags\",\n \"elasticloadbalancing:DescribeTargetGroupAttributes\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteSnapshot\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddListenerCertificates\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteRule\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyRule\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:ModifyTargetGroupAttributes\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:RemoveListenerCertificates\",\n \"elasticloadbalancing:RemoveTags\",\n \"elasticloadbalancing:SetIpAddressType\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\",\n \"elasticloadbalancing:SetSecurityGroups\",\n \"elasticloadbalancing:SetSubnets\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateRule\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:37.930193 26677 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0528 04:37:37.930414 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0528 04:37:37.930642 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.930787 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.930999 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.931152 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" }] }] } I0528 04:37:37.931361 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.931591 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.931771 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.931860 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.931934 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.932150 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:37.932318 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.932489 26677 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.932669 26677 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:37.932885 26677 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0528 04:37:37.933074 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.933443 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0528 04:37:37.933635 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.933745 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.933846 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.933937 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.934045 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.934254 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.934404 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0528 04:37:37.934456 26677 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0528 04:37:37.934496 26677 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0528 04:37:37.936465 26677 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0528 04:37:37.936750 26677 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0528 04:37:37.937039 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:37.937107 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:37.937150 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:37.937212 26677 dns.go:232] Pre-creating DNS records I0528 04:37:37.937242 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:37.943186 26677 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0528 04:37:37.946786 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.946878 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:37.985166 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:38.689533 26677 iamrole.go:89] ListRoles: { } I0528 04:37:38.690567 26677 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0528 04:37:38.691978 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.692340 26677 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0528 04:37:38.692629 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.693425 26677 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0528 04:37:38.693776 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.694389 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0528 04:37:38.694545 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:38.694894 26677 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:38.695059 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:38.697942 26677 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0528 04:37:38.698158 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0528 04:37:38.698365 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0528 04:37:38.698428 26677 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:38.698499 26677 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:38.698442 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.698665 26677 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:38.698719 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:38.698891 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.699104 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0528 04:37:38.699392 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:38.699525 26677 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:38.699621 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.701337 26677 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0528 04:37:38.701469 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:38.701611 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:38.701700 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:38.701784 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:38.701859 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:38.701977 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:38.702275 26677 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0528 04:37:38.704676 26677 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0528 04:37:38.704940 26677 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0528 04:37:38.705187 26677 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0528 04:37:38.708553 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0528 04:37:38.708615 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0528 04:37:38.708656 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0528 04:37:38.708710 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0528 04:37:38.708750 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0528 04:37:38.708798 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0528 04:37:38.708838 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0528 04:37:38.708882 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0528 04:37:38.708933 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0528 04:37:38.708976 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0528 04:37:38.709033 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0528 04:37:38.709071 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0528 04:37:38.709117 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0528 04:37:38.709168 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0528 04:37:38.709203 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:38.709254 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0528 04:37:38.709294 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:38.709349 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0528 04:37:38.709397 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0528 04:37:38.709429 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0528 04:37:38.709485 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0528 04:37:38.709541 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0528 04:37:38.709591 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:38.709633 26677 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0528 04:37:38.711312 26677 delete_cluster.go:128] Looking for cloud resources to delete W0528 04:37:38.711386 26677 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0528 04:37:38.711409 26677 keypairs.go:95] DescribeKeyPairs: { } I0528 04:37:38.711461 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711528 26677 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0528 04:37:38.711618 26677 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711735 26677 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711791 26677 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711877 26677 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711932 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.711990 26677 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0528 04:37:38.712048 26677 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.712123 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0528 04:37:38.712171 26677 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0528 04:37:38.712241 26677 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0528 04:37:38.712281 26677 api.go:53] PageSize not implemented I0528 04:37:38.712302 26677 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0528 04:37:38.712342 26677 loadbalancers.go:35] PageSize not implemented I0528 04:37:38.712369 26677 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0528 04:37:38.712398 26677 targetgroups.go:35] PageSize not implemented I0528 04:37:38.712416 26677 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0528 04:37:38.712495 26677 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0528 04:37:38.712548 26677 iamrole.go:89] ListRoles: { } I0528 04:37:38.712605 26677 zones.go:93] ListHostedZonesPages { } I0528 04:37:38.712638 26677 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0528 04:37:38.712697 26677 internetgateways.go:101] DescribeInternetGateways: { } I0528 04:37:38.712750 26677 routetable.go:53] DescribeRouteTables: { } I0528 04:37:38.712797 26677 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0528 04:37:38.713250 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-config:lt-1 ok I0528 04:37:38.713321 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok I0528 04:37:38.713337 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0528 04:37:38.713394 26677 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:38.713474 26677 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0528 04:37:38.713425 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0528 04:37:38.713587 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:38.713615 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0528 04:37:38.713653 26677 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0528 04:37:38.713493 26677 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0528 04:37:38.713712 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0528 04:37:38.713772 26677 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0528 04:37:38.713834 26677 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0528 04:37:38.713867 26677 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0528 04:37:38.713918 26677 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0528 04:37:38.714162 26677 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0528 04:37:38.714181 26677 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0528 04:37:38.714213 26677 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0528 04:37:38.714314 26677 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0528 04:37:38.714362 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0528 04:37:38.714455 26677 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0528 04:37:38.714474 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0528 04:37:38.714509 26677 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0528 04:37:38.714544 26677 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0528 04:37:38.714590 26677 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0528 04:37:38.714602 26677 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0528 04:37:38.714628 26677 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0528 04:37:38.714685 26677 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0528 04:37:38.714757 26677 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0528 04:37:38.714823 26677 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0528 04:37:38.715123 26677 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleManyAddons (1.55s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 56.389s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.006s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0528 04:36:42.346913 26683 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.005s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.006s === RUN TestPodController I0528 04:36:42.376401 26692 pod.go:59] starting pod controller I0528 04:36:42.376709 26692 stoppable.go:62] shutting down controller I0528 04:36:42.376764 26692 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.038s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a7f70} route53_test.go:93: Got zone list: [0xc0000a7fa0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc0000120c0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039e0d0} route53_test.go:93: Got zone list: [0xc00039e100] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc0000120d8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00039e200} route53_test.go:170: Successfully added managed DNS zone: &{0xc00039a150 0xc0000120f8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039e3c0} route53_test.go:93: Got zone list: [0xc00039e3f0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012118} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039e550} route53_test.go:93: Got zone list: [0xc00039e580] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012140} route53_test.go:187: Successfully added resource record set: {0xc000182600 0xc000012160} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039e860} route53_test.go:93: Got zone list: [0xc00039e890] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc0000121f0} route53_test.go:199: Successfully added resource record set: {0xc000182780 0xc000012210} route53_test.go:118: Got 1 recordsets: [0xc00039ea30] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039ebc0} route53_test.go:93: Got zone list: [0xc00039ebf0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc0000122b0} route53_test.go:221: Successfully added resource record set: {0xc000182900 0xc0000122d0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000182900 0xc0000122d0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039efb0} route53_test.go:93: Got zone list: [0xc00039efe0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012390} route53_test.go:246: Successfully removed resource set {0xc000182b00 0xc0000123b0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039f2c0} route53_test.go:93: Got zone list: [0xc00039f2f0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012440} route53_test.go:264: Successfully removed resource set {0xc000182c80 0xc000012460} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039f600} route53_test.go:93: Got zone list: [0xc00039f630] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc0000124f8} commontests.go:192: Successfully added resource record set: {0xc000182e00 0xc000012518} commontests.go:45: Correctly replaced resource record {0xc000182e00 0xc000012518} -> {0xc000182f00 0xc000012568} commontests.go:142: Got 1 recordsets: [0xc00039f940] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00039fb70} route53_test.go:93: Got zone list: [0xc00039fba0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012620} commontests.go:192: Successfully added resource record set: {0xc000183180 0xc000012640} commontests.go:70: Correctly replaced resource record {0xc000183180 0xc000012640} -> {0xc000183280 0xc000012690} commontests.go:142: Got 1 recordsets: [0xc00039feb0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ac130} route53_test.go:93: Got zone list: [0xc0003ac160] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012758} commontests.go:192: Successfully added resource record set: {0xc000183500 0xc000012778} commontests.go:142: Got 2 recordsets: [0xc0003ac3d0 0xc0003ac3f0] commontests.go:142: Got 2 recordsets: [0xc0003ac460 0xc0003ac480] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ac680} route53_test.go:93: Got zone list: [0xc0003ac6b0] route53_test.go:98: Got at least 1 zone in list:&{0xc00036bef0 0xc000012880} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.016s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.008s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000488b40} clouddns_test.go:71: Got zone list: [0xc0005246d8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc0005246a8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000488c40} clouddns_test.go:71: Got zone list: [0xc000524798] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000524768} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000488d40} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00048e6c0 0xc000524858} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000488e00} clouddns_test.go:71: Got zone list: [0xc000524918] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc0005248e8} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000488f20} clouddns_test.go:71: Got zone list: [0xc000524a50] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000524a20} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000489060} clouddns_test.go:71: Got zone list: [0xc000524bb8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000524b88} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc0004891c0} clouddns_test.go:71: Got zone list: [0xc000524dc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000524d98} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000489320} clouddns_test.go:71: Got zone list: [0xc000524f60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000524f30} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000489460} clouddns_test.go:71: Got zone list: [0xc0005250c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000525098} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc0004895c0} clouddns_test.go:71: Got zone list: [0xc000525290] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000525260} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000489720} clouddns_test.go:71: Got zone list: [0xc000525590] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000525560} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc0004898a0} clouddns_test.go:71: Got zone list: [0xc000525a28] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc0005259f8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00048afb0 0xc000489a80} clouddns_test.go:71: Got zone list: [0xc000525e00] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00048e540 0xc000525dd0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.063s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.026s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.058s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0528 04:36:50.363446 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.363758 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.363872 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.363972 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.364069 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.364206 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.364309 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.364368 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.364462 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.364530 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.382929 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0528 04:36:50.387332 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.387454 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.387483 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.387516 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.387543 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.387574 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.387601 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.387619 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.387642 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.387659 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.388871 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0528 04:36:50.392556 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.392628 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.392653 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.392677 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.392703 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.392731 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.392764 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.392787 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.392808 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.392837 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.393901 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.395523 27270 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0528 04:36:50.395541 27270 assetstore.go:111] bin/containerd I0528 04:36:50.395550 27270 assetstore.go:111] bin/containerd-shim I0528 04:36:50.395557 27270 assetstore.go:111] bin/containerd-shim-runc-v1 I0528 04:36:50.395564 27270 assetstore.go:111] bin/containerd-shim-runc-v2 I0528 04:36:50.395571 27270 assetstore.go:111] bin/containerd-stress I0528 04:36:50.395578 27270 assetstore.go:111] bin/ctr I0528 04:36:50.395614 27270 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0528 04:36:50.395643 27270 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0528 04:36:50.399098 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.399173 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.399199 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.399225 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.399249 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.399278 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.399305 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.399328 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.399351 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.399368 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.400377 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.401794 27270 containerd.go:60] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0528 04:36:50.402734 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.402808 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.402830 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.402850 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.402872 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.402893 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.402912 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.402927 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.402944 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.402959 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.403830 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.405395 27270 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0528 04:36:50.406103 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.406187 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.406211 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.406244 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.406276 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.406309 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.406347 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.406371 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.406397 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.406416 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.407837 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.409418 27270 assetstore.go:108] Matching assets for "^docker/": I0528 04:36:50.409438 27270 assetstore.go:111] docker/containerd I0528 04:36:50.409446 27270 assetstore.go:111] docker/containerd-shim I0528 04:36:50.409453 27270 assetstore.go:111] docker/ctr I0528 04:36:50.409466 27270 assetstore.go:111] docker/docker I0528 04:36:50.409475 27270 assetstore.go:111] docker/docker-init I0528 04:36:50.409483 27270 assetstore.go:111] docker/docker-proxy I0528 04:36:50.409490 27270 assetstore.go:111] docker/dockerd I0528 04:36:50.409496 27270 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0528 04:36:50.412309 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.412389 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.412411 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.412431 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.412449 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.412470 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.412488 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.412514 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.412533 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.412550 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.413511 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.414828 27270 assetstore.go:108] Matching assets for "^docker/": I0528 04:36:50.414843 27270 assetstore.go:111] docker/containerd I0528 04:36:50.414850 27270 assetstore.go:111] docker/containerd-shim I0528 04:36:50.414856 27270 assetstore.go:111] docker/ctr I0528 04:36:50.414862 27270 assetstore.go:111] docker/docker I0528 04:36:50.414868 27270 assetstore.go:111] docker/docker-init I0528 04:36:50.414874 27270 assetstore.go:111] docker/docker-proxy I0528 04:36:50.414880 27270 assetstore.go:111] docker/dockerd I0528 04:36:50.414886 27270 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0528 04:36:50.419841 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.419949 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.419980 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.420016 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.420054 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.420098 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.420131 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.420156 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.420186 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.420207 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.421538 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.423518 27270 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0528 04:36:50.423652 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.423724 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.423747 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.423777 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.423806 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.423840 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.423867 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.423892 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.423921 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.423940 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.425210 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:50.427028 27270 assetstore.go:108] Matching assets for "^docker/": I0528 04:36:50.427048 27270 assetstore.go:111] docker/docker I0528 04:36:50.427074 27270 assetstore.go:111] docker/docker-containerd I0528 04:36:50.427088 27270 assetstore.go:111] docker/docker-containerd-ctr I0528 04:36:50.427099 27270 assetstore.go:111] docker/docker-containerd-shim I0528 04:36:50.427110 27270 assetstore.go:111] docker/docker-init I0528 04:36:50.427120 27270 assetstore.go:111] docker/docker-proxy I0528 04:36:50.427130 27270 assetstore.go:111] docker/docker-runc I0528 04:36:50.427140 27270 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0528 04:36:50.433552 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.433672 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.433713 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.433757 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.433802 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.433850 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.433891 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.433927 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.433969 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.433999 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.438202 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0528 04:36:50.442034 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.442154 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.442192 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.442235 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.442273 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.442313 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.442351 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.442379 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.442414 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.442438 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.445446 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0528 04:36:50.449182 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.449284 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.449316 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.449355 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.449394 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.449435 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.449472 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.449502 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.449541 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.449567 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.452230 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0528 04:36:50.458404 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.458500 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.458523 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.458546 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.458567 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.458593 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.458616 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.458635 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.458656 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.458669 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.460365 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0528 04:36:50.470539 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.470624 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.470650 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.470685 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.470716 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.470742 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.470765 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.470782 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.470802 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.470817 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.472596 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0528 04:36:50.479079 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.479167 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.479192 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.479218 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.479240 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.479268 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.479293 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.479310 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.479333 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.479348 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.480933 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0528 04:36:50.487350 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.487422 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.487443 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.487468 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.487491 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.487517 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.487539 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.487566 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.487590 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.487604 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.489305 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0528 04:36:50.498717 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:50.498811 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:50.498842 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:50.498882 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:50.498917 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:50.498963 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:50.498995 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:50.499030 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:50.499070 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:50.499094 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:50.500800 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.49s) === RUN TestKubeAPIServerBuilderARM64 I0528 04:36:51.990752 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:51.990881 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:51.990919 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:51.990963 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:51.991001 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:51.991048 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:51.991087 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:51.991117 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:51.991154 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:51.991177 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:51.994076 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0528 04:36:52.073869 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.074050 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.074097 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.074141 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.074180 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.074225 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.074266 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.074294 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.074328 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.074354 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.077359 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0528 04:36:52.088353 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.088474 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.088511 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.088551 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.088588 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.088635 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.088673 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.088700 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.088737 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.088761 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.091639 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0528 04:36:52.162311 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.162424 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.162467 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.162516 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.162556 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.162606 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.162645 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.162673 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.162708 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.162726 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.164655 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.09s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0528 04:36:52.256646 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.256779 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.256822 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.256878 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.256925 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.256980 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.257030 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.257063 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.257103 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.257133 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.260306 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0528 04:36:52.266700 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.266831 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.266873 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.266920 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.266963 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.267016 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.267065 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.267098 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.267139 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.267169 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.270152 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.08s) === RUN TestKubeProxyBuilderARM64 I0528 04:36:52.343416 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.343548 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.343585 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.343631 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.343670 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.343713 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.343753 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.343781 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.343815 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.343841 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.346713 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.08s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0528 04:36:52.419822 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.419920 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.419950 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.419979 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.420003 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.420034 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.420061 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.420079 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.420103 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.420121 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.421985 27270 zones.go:93] ListHostedZonesPages { } W0528 04:36:52.424550 27270 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0528 04:36:52.426574 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.426668 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.426696 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.426723 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.426746 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.426775 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.426799 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.426818 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.426841 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.426857 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.428779 27270 zones.go:93] ListHostedZonesPages { } W0528 04:36:52.498143 27270 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0528 04:36:52.501173 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.501301 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.501340 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.501388 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.501429 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.501477 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.501518 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.501551 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.501604 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.501634 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.504745 27270 zones.go:93] ListHostedZonesPages { } W0528 04:36:52.570007 27270 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0528 04:36:52.571826 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.571913 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.571943 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.571977 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.572003 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.572035 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.572061 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.572078 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.572103 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.572122 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.573809 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:52.575595 27270 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0528 04:36:52.577348 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.577408 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.577443 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.577469 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.577492 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.577522 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.577543 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.577561 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.577582 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.577597 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.578877 27270 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I0528 04:36:52.582187 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.582247 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.582283 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.582316 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.582338 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.582365 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.582396 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.582416 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.582436 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.582451 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.583629 27270 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0528 04:36:52.586805 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.586879 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.586902 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.586926 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.586950 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.586976 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.586998 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.587017 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.587038 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.587053 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.588778 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:52.590406 27270 assetstore.go:108] Matching assets for "protokube$": I0528 04:36:52.590424 27270 assetstore.go:111] /opt/kops/bin/protokube I0528 04:36:52.590444 27270 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0528 04:36:52.590467 27270 assetstore.go:108] Matching assets for "channels$": I0528 04:36:52.590478 27270 assetstore.go:111] /opt/kops/bin/channels I0528 04:36:52.590485 27270 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0528 04:36:52.592899 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.593464 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.593500 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.593533 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.593561 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.593592 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.593619 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.593638 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.593662 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.593681 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.595599 27270 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0528 04:36:52.597937 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.598024 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.598050 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.598078 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.598104 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.598134 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.598160 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.598177 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.598200 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.598217 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.599963 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:52.601636 27270 update_service.go:91] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0528 04:36:52.602069 27270 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:36:52.602150 27270 internetgateways.go:66] CreateInternetGateway: { } I0528 04:36:52.602175 27270 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:36:52.602204 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:36:52.602230 27270 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:36:52.602259 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:36:52.602283 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:36:52.602301 27270 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:36:52.602325 27270 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:36:52.602341 27270 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:36:52.604020 27270 zones.go:93] ListHostedZonesPages { } I0528 04:36:52.605699 27270 update_service.go:86] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.324s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.022s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.027s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.022s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.022s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0528 04:36:50.298517 27276 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0528 04:36:50.298625 27276 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.010s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0528 04:36:50.893237 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893323 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893355 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893378 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893414 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0528 04:36:50.893637 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893680 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893708 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893735 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893766 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893792 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893826 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893852 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893882 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893913 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893940 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.893973 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894000 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894023 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894048 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894088 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894114 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894140 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0528 04:36:50.894164 27627 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0528 04:36:50.894330 27627 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0528 04:36:50.894393 27627 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.076s === RUN TestManagedFieldsMigrator === RUN TestManagedFieldsMigrator/csa-migration === RUN TestManagedFieldsMigrator/merge-owners === RUN TestManagedFieldsMigrator/ssa-unchanged === RUN TestManagedFieldsMigrator/unknown-unchanged --- PASS: TestManagedFieldsMigrator (0.01s) --- PASS: TestManagedFieldsMigrator/csa-migration (0.00s) --- PASS: TestManagedFieldsMigrator/merge-owners (0.00s) --- PASS: TestManagedFieldsMigrator/ssa-unchanged (0.00s) --- PASS: TestManagedFieldsMigrator/unknown-unchanged (0.00s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.024s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0528 04:36:50.953861 27629 builder.go:230] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.10s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0528 04:36:50.955746 27629 builder.go:230] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0528 04:36:50.956916 27629 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0528 04:36:50.957778 27629 builder.go:230] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0528 04:36:50.958749 27629 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0528 04:36:50.959502 27629 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0528 04:36:50.960229 27629 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0528 04:36:50.961088 27629 copyfile.go:208] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0528 04:36:50.961109 27629 copyfile.go:209] S3, Google Cloud Storage, and File Paths are supported. E0528 04:36:50.961120 27629 copyfile.go:210] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0528 04:36:50.961132 27629 copyfile.go:211] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.133s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.004s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.066s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.020s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.087s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.021s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.005s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.004s === RUN TestHasExtraFields === RUN TestHasExtraFields/minimal === RUN TestHasExtraFields/extraFields === RUN TestHasExtraFields/spec2 === RUN TestHasExtraFields/isolateMasters === RUN TestHasExtraFields/instanceGroup --- PASS: TestHasExtraFields (0.02s) --- PASS: TestHasExtraFields/minimal (0.01s) --- PASS: TestHasExtraFields/extraFields (0.00s) --- PASS: TestHasExtraFields/spec2 (0.00s) --- PASS: TestHasExtraFields/isolateMasters (0.00s) --- PASS: TestHasExtraFields/instanceGroup (0.00s) PASS ok k8s.io/kops/pkg/edit 0.041s === RUN TestFlagToFalse I0528 04:36:58.060972 28784 featureflag_test.go:32] Created flag Unittest1 I0528 04:36:58.061018 28784 featureflag.go:166] FeatureFlag "UnitTest1"=false I0528 04:36:58.061028 28784 featureflag.go:166] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0528 04:36:58.061090 28784 featureflag.go:166] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) === RUN TestGetPositive --- PASS: TestGetPositive (0.00s) === RUN TestGetNegative --- PASS: TestGetNegative (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.004s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.031s === RUN TestRenderInstanceGroupZones W0528 04:36:58.076923 28786 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.021s === RUN TestWarmPoolOnlyRoll I0528 04:37:11.727193 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.727273 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.727290 29266 instancegroups.go:136] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0528 04:37:11.727661 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.727682 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.728871 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.728912 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.734422 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.734463 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.734486 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.734497 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.736208 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.736760 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.737229 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.737265 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.738352 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.738419 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.744147 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.744231 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:11.745248 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:11.745331 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:11.745369 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.746500 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.746533 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.751857 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.751892 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.751915 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.751932 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.752962 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.753767 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.753804 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.757148 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.757193 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.762440 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.762512 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:11.763539 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.763581 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.764736 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.764761 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.769980 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.770055 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:11.771042 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.771081 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.772221 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.772262 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.777482 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.777521 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.777546 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.777565 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:11.778732 29266 instancegroups.go:431] Draining the node: "node-2a.local". I0528 04:37:11.779565 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:11.779606 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.780743 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.780776 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.786001 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.786071 29266 instancegroups.go:431] Draining the node: "node-2b.local". I0528 04:37:11.787143 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:11.787183 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.788300 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.788333 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.793566 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.793647 29266 instancegroups.go:431] Draining the node: "node-2c.local". I0528 04:37:11.794684 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:11.794729 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.795870 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.795903 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.801148 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.801189 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0528 04:37:11.801644 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.801671 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.802837 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.802897 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.802950 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.802970 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.803002 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.804118 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.804154 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.804165 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:11.804183 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.805265 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.805288 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.805328 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.805341 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.805360 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.806490 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.806548 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.806567 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.806597 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.807731 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.807773 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.807789 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.807816 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.808917 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.808934 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.808973 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.808986 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:11.809003 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.810117 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.810158 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.810181 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:11.810217 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.811316 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0528 04:37:11.811338 29266 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0528 04:37:11.811346 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:11.811379 29266 instancegroups.go:467] waiting for 1ms after terminating instance W0528 04:37:11.812525 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0528 04:37:11.812559 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0528 04:37:11.812941 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.812963 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.814088 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.814131 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.814153 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.814195 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.814214 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.814230 29266 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.814244 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.814964 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.815545 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.815606 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.815627 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.816743 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.816769 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.816785 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.816811 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:11.817451 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:11.817501 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:11.817530 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.818678 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.818706 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.818721 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.818737 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.818751 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.818760 29266 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.818775 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.819425 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.819974 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.819999 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.821101 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.821125 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.821139 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.821168 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:11.821752 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.821779 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.822919 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.822959 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.822983 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.823027 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:11.823891 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.823927 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.825050 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.825081 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.825103 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.825124 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.825148 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.825166 29266 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.825184 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:11.825979 29266 instancegroups.go:431] Draining the node: "node-2a.local". I0528 04:37:11.826484 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:11.826509 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.827635 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.827676 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.827698 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.827738 29266 instancegroups.go:431] Draining the node: "node-2b.local". I0528 04:37:11.828580 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:11.828611 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.830031 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.830075 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.830095 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.830130 29266 instancegroups.go:431] Draining the node: "node-2c.local". I0528 04:37:11.830725 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:11.830751 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.831887 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.831950 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0528 04:37:11.831979 29266 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0528 04:37:11.832009 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0528 04:37:11.832314 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0528 04:37:11.832578 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.832608 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.833767 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.833790 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.839000 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.839086 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.839106 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.839118 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.839719 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.840247 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.840308 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.840334 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.841497 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.841540 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.846759 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.846819 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:11.847519 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:11.847579 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:11.847606 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.848757 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.848797 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.853990 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.854027 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.854044 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.854055 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.854755 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.855316 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.855362 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.856521 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.856568 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.861787 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.861855 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:11.862493 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.862517 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.863678 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.863711 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.868897 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.868930 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:11.869425 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.869446 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.870605 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.870642 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.875883 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.875919 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.875942 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.875956 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:11.876808 29266 instancegroups.go:431] Draining the node: "node-2a.local". I0528 04:37:11.877309 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:11.877336 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.880267 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.880314 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.885525 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.885586 29266 instancegroups.go:431] Draining the node: "node-2b.local". I0528 04:37:11.886614 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:11.886649 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.887781 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.887809 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.893019 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.893077 29266 instancegroups.go:431] Draining the node: "node-2c.local". I0528 04:37:11.893723 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:11.893748 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.894871 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.894898 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.900117 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.900164 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0528 04:37:11.900362 29266 rollingupdate.go:111] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0528 04:37:11.900855 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.900886 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.902024 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.902059 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.902080 29266 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0528 04:37:11.902414 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.902443 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.903859 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.903887 29266 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0528 04:37:11.903910 29266 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0528 04:37:11.904152 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.904179 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.904199 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.904217 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0528 04:37:11.904381 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.904404 29266 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0528 04:37:11.904424 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.904444 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0528 04:37:11.904675 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.904705 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.905839 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.905868 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.911096 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.911179 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.911207 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.911225 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.912054 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.912793 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.912866 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.912899 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.914030 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.914066 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.914087 29266 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0528 04:37:11.914422 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.914452 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.915606 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.915635 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.920859 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.920943 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.920992 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.921006 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.921623 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.922089 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.922142 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.922162 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.923285 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.923320 29266 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0528 04:37:11.923341 29266 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0528 04:37:11.924394 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.924419 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.924432 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.925511 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.926273 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.926308 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.927435 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.927460 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.932676 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.932745 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:11.933696 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.933732 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.934865 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.934896 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.940120 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.940190 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:11.941151 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.941191 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.942891 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.942920 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.948898 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.948948 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0528 04:37:11.949277 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.949305 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.949324 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.949338 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0528 04:37:11.949541 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.949564 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.949578 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.949591 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0528 04:37:11.949789 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.949812 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.949827 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.949840 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupExitableError I0528 04:37:11.950070 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.950091 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.950103 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.951087 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.951767 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.951804 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.953313 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.953345 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.958565 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.958639 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:11.959602 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:11.959642 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.960793 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.960824 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.966011 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.966084 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:11.966718 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:11.966742 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.967889 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.967911 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.973110 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.973145 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.973165 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.973183 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.973193 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-2": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupExitableError (0.02s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0528 04:37:11.973421 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.973438 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.973447 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.974137 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.974616 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.974638 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.975749 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.975776 29266 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0528 04:37:11.975788 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.975799 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0528 04:37:11.975924 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.975940 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.975948 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:11.976486 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:11.976900 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:11.976919 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.978057 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.978081 29266 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0528 04:37:11.978095 29266 instancegroups.go:506] Cluster did not validate within 0s E0528 04:37:11.978105 29266 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0528 04:37:11.978319 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:11.978338 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:11.979480 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.979508 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:11.980622 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.985829 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:11.986968 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.992195 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:11.993326 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:11.998555 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.998635 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:11.998663 29266 instancegroups.go:537] Cluster validated. I0528 04:37:11.998682 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:11.999206 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:11.999672 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:11.999733 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:11.999761 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.000989 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.001040 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.006252 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.006322 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:12.007151 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:12.007220 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:12.007251 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.008424 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.008478 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.013732 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.013783 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.013825 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.013844 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:12.014842 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.015584 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.015617 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.016843 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.016890 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.022102 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.022167 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.023015 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.023049 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.024163 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.024201 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.029409 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.029479 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:12.030421 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:12.030458 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.031574 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.031627 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.036854 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.036894 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.036926 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.036945 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:12.038038 29266 instancegroups.go:431] Draining the node: "node-2a.local". I0528 04:37:12.038873 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:12.038910 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.040065 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.040099 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.045321 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.045390 29266 instancegroups.go:431] Draining the node: "node-2b.local". I0528 04:37:12.046307 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:12.046345 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.047664 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.047714 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.052945 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.053017 29266 instancegroups.go:431] Draining the node: "node-2c.local". I0528 04:37:12.053951 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:12.053986 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.055128 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.055177 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.060509 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.060558 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0528 04:37:12.060989 29266 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0528 04:37:12.061023 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.062166 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.062206 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:12.063326 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:12.064446 29266 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0528 04:37:12.065565 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.070785 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.070875 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.070906 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.070927 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:12.071724 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:12.072317 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:12.072384 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:12.072418 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.073597 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.073634 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.078841 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.078913 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:12.079736 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:12.079791 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:12.079815 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.080963 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.080986 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.086184 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.086235 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.086250 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.086262 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:12.086934 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.087570 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.087597 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.088746 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.088786 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.093974 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.094035 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.094661 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.094688 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.095819 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.095854 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.101012 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.101063 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:12.101710 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:12.101737 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.102900 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.102939 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.108390 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.108435 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.108459 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.108475 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:12.109297 29266 instancegroups.go:431] Draining the node: "node-2a.local". I0528 04:37:12.109838 29266 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0528 04:37:12.109872 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.111041 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.111089 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.117090 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.117196 29266 instancegroups.go:431] Draining the node: "node-2b.local". I0528 04:37:12.117900 29266 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0528 04:37:12.117935 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.119057 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.119083 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.124297 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.124365 29266 instancegroups.go:431] Draining the node: "node-2c.local". I0528 04:37:12.125223 29266 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0528 04:37:12.125252 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.126381 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.126419 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.131635 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.131690 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0528 04:37:12.132266 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.132284 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.132293 29266 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0528 04:37:12.133028 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.133578 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.133602 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.134724 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.134753 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.139965 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.140026 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.141245 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.141311 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.142351 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.142380 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.147953 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.147994 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0528 04:37:12.148317 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.148337 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.148345 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:12.148872 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:12.149382 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:12.149434 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:12.149453 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.150626 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.150661 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.155818 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.155891 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:12.156786 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:12.156854 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:12.156886 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.158025 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.158056 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.163667 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.163712 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0528 04:37:12.163992 29266 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled W0528 04:37:12.164023 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0528 04:37:12.164037 29266 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled W0528 04:37:12.164047 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0528 04:37:12.164058 29266 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled W0528 04:37:12.164065 29266 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0528 04:37:12.164076 29266 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0528 04:37:12.164089 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0528 04:37:12.164234 29266 instancegroups.go:604] Detaching instance "bastion-1a", in group "bastion-1". I0528 04:37:12.164269 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:12.165391 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.165420 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.177470 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.177525 29266 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled I0528 04:37:12.177615 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.177656 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.177671 29266 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0528 04:37:12.178411 29266 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled I0528 04:37:12.178433 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.178451 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.178463 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:12.179291 29266 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0528 04:37:12.179349 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:12.180464 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.180492 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.185688 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.185731 29266 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled I0528 04:37:12.185751 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.185766 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.185778 29266 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0528 04:37:12.186491 29266 instancegroups.go:602] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0528 04:37:12.186537 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:12.187687 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.187708 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:12.192893 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.192930 29266 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0528 04:37:12.192952 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.03s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0528 04:37:12.193233 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.193257 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.193267 29266 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0528 04:37:12.194722 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.195293 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.195327 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.196470 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.196499 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.196529 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:12.196543 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.197099 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:12.197525 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.197551 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.198687 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.198730 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.299184 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.299252 29266 instancegroups.go:431] Draining the node: "node-1e.local". I0528 04:37:12.299326 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:12.300196 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:12.300356 29266 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0528 04:37:12.300386 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.301533 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.301600 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.301643 29266 instancegroups.go:431] Draining the node: "node-1f.local". I0528 04:37:12.302450 29266 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0528 04:37:12.302489 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.303644 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.303689 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.404119 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.404183 29266 instancegroups.go:431] Draining the node: "node-1g.local". I0528 04:37:12.404796 29266 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0528 04:37:12.404832 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.406002 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.406050 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.406066 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0528 04:37:12.406277 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.406297 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.406307 29266 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0528 04:37:12.407386 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.407431 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.407859 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.408204 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.408230 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.409381 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.409421 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.509841 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.509910 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:12.509960 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:12.510541 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:12.511038 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:12.511073 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.512228 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.512258 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.512281 29266 instancegroups.go:431] Draining the node: "node-1e.local". I0528 04:37:12.512817 29266 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0528 04:37:12.512844 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.513946 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.513982 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.614363 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.614444 29266 instancegroups.go:431] Draining the node: "node-1f.local". I0528 04:37:12.615423 29266 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0528 04:37:12.615477 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.616619 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.616659 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.616681 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0528 04:37:12.616940 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.616968 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.616982 29266 instancegroups.go:342] Tainting 7 nodes in "master-1" instancegroup. I0528 04:37:12.619013 29266 instancegroups.go:431] Draining the node: "master-1a.local". I0528 04:37:12.619712 29266 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0528 04:37:12.619780 29266 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0528 04:37:12.619822 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.621000 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.621054 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.621097 29266 instancegroups.go:431] Draining the node: "master-1c.local". I0528 04:37:12.621670 29266 instancegroups.go:431] Draining the node: "master-1b.local". I0528 04:37:12.621902 29266 instancegroups.go:688] skipping deregistration of instance "master-1c", as part of instancegroup with role "ControlPlane" I0528 04:37:12.621971 29266 instancegroups.go:625] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0528 04:37:12.622480 29266 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0528 04:37:12.622537 29266 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0528 04:37:12.622571 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.623711 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.623781 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.724203 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.724289 29266 instancegroups.go:431] Draining the node: "master-1e.local". I0528 04:37:12.725154 29266 instancegroups.go:688] skipping deregistration of instance "master-1e", as part of instancegroup with role "ControlPlane" I0528 04:37:12.725225 29266 instancegroups.go:625] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0528 04:37:12.725276 29266 instancegroups.go:431] Draining the node: "master-1d.local". I0528 04:37:12.725963 29266 instancegroups.go:688] skipping deregistration of instance "master-1d", as part of instancegroup with role "ControlPlane" I0528 04:37:12.726012 29266 instancegroups.go:625] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0528 04:37:12.726050 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.733970 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.734043 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.734094 29266 instancegroups.go:431] Draining the node: "master-1f.local". I0528 04:37:12.734948 29266 instancegroups.go:688] skipping deregistration of instance "master-1f", as part of instancegroup with role "ControlPlane" I0528 04:37:12.735023 29266 instancegroups.go:625] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0528 04:37:12.735062 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.736200 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.736260 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.836652 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.836721 29266 instancegroups.go:431] Draining the node: "master-1g.local". I0528 04:37:12.837588 29266 instancegroups.go:688] skipping deregistration of instance "master-1g", as part of instancegroup with role "ControlPlane" I0528 04:37:12.837656 29266 instancegroups.go:625] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0528 04:37:12.837699 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.838842 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.838890 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.838917 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0528 04:37:12.839202 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.839237 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.839251 29266 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0528 04:37:12.840903 29266 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0528 04:37:12.840965 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:12.842084 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.842115 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.842126 29266 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0528 04:37:12.842172 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:12.843286 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.843316 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.843356 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:12.843383 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:12.844394 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:12.844653 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:12.844698 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.845819 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.845862 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.946258 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.946327 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:12.946354 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:12.947245 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:12.947485 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:12.947526 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.948688 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.948720 29266 instancegroups.go:537] Cluster validated. I0528 04:37:12.948744 29266 instancegroups.go:431] Draining the node: "node-1e.local". I0528 04:37:12.949260 29266 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0528 04:37:12.949294 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:12.950435 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:12.950480 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.050952 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.051014 29266 instancegroups.go:431] Draining the node: "node-1f.local". I0528 04:37:13.051634 29266 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0528 04:37:13.051677 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.052826 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.052855 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.052870 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0528 04:37:13.053082 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.053102 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.053111 29266 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0528 04:37:13.054588 29266 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0528 04:37:13.054632 29266 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0528 04:37:13.054675 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:13.055796 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.055823 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.055851 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.055873 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.056718 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.056874 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.056899 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.058025 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.058059 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.158479 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.158541 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:13.158581 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:13.159390 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:13.159704 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:13.159751 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.160902 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.160931 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.160953 29266 instancegroups.go:431] Draining the node: "node-1e.local". I0528 04:37:13.161465 29266 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0528 04:37:13.161496 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.162607 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.162653 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.263319 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.263380 29266 instancegroups.go:431] Draining the node: "node-1f.local". I0528 04:37:13.264005 29266 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0528 04:37:13.264046 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.265209 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.265239 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.265253 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0528 04:37:13.265430 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.265445 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.265453 29266 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0528 04:37:13.265882 29266 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0528 04:37:13.265903 29266 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0528 04:37:13.265924 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:13.267062 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.267081 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.272268 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.272333 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.272343 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.273364 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.273400 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.273604 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.273636 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.274551 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.274572 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.279781 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.279812 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0528 04:37:13.280063 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.280085 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.280100 29266 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0528 04:37:13.280773 29266 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0528 04:37:13.280829 29266 instancegroups.go:191] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0528 04:37:13.280845 29266 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0528 04:37:13.280886 29266 instancegroups.go:191] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0528 04:37:13.280919 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:13.282059 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.282078 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.287265 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.287319 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.287326 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.288238 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.288279 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.288491 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.288522 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.289400 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.289419 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.294623 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.294646 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0528 04:37:13.294869 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.294896 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.294910 29266 instancegroups.go:342] Tainting 4 nodes in "node-1" instancegroup. I0528 04:37:13.295930 29266 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0528 04:37:13.295971 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:13.297107 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.297135 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.297143 29266 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0528 04:37:13.297174 29266 instancegroups.go:203] waiting for 1ms after detaching instance I0528 04:37:13.298279 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.298303 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.298331 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:13.298345 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.298390 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.298801 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:13.298827 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.299489 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.299519 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.299676 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.299702 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.299924 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.400369 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.400429 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:13.401031 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:13.401068 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.402210 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.402243 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.402258 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0528 04:37:13.402439 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.402458 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.402467 29266 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0528 04:37:13.403697 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.404111 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.404137 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.405271 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.405307 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.405348 29266 instancegroups.go:431] Draining the node: "node-1e.local". I0528 04:37:13.405595 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:13.405946 29266 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0528 04:37:13.406375 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:13.406410 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.407560 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.407601 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.508032 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.508094 29266 instancegroups.go:431] Draining the node: "node-1g.local". I0528 04:37:13.508142 29266 instancegroups.go:431] Draining the node: "node-1f.local". I0528 04:37:13.509120 29266 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0528 04:37:13.509391 29266 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0528 04:37:13.509441 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.510602 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.510630 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.510652 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.511128 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.511156 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.512279 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.512336 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.612843 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.612902 29266 instancegroups.go:431] Draining the node: "node-1d.local". I0528 04:37:13.613509 29266 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0528 04:37:13.613542 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.614707 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.614753 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.614770 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0528 04:37:13.615023 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.615041 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.615050 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0528 04:37:13.615063 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0528 04:37:13.615072 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0528 04:37:13.615087 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0528 04:37:13.615184 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.615197 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.615205 29266 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0528 04:37:13.615824 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0528 04:37:13.615841 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0528 04:37:13.615852 29266 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0528 04:37:13.615877 29266 instancegroups.go:431] Draining the node: "node-1a.local". I0528 04:37:13.616342 29266 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0528 04:37:13.616362 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.617512 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.617535 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.622718 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.622772 29266 instancegroups.go:431] Draining the node: "node-1b.local". I0528 04:37:13.623371 29266 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0528 04:37:13.623399 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.624926 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.624960 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.630468 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.630536 29266 instancegroups.go:431] Draining the node: "node-1c.local". I0528 04:37:13.631315 29266 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0528 04:37:13.631350 29266 instancegroups.go:467] waiting for 1ms after terminating instance I0528 04:37:13.632525 29266 instancegroups.go:501] Validating the cluster. I0528 04:37:13.632559 29266 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0528 04:37:13.637806 29266 instancegroups.go:537] Cluster validated. I0528 04:37:13.637842 29266 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.026s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.004s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.016s === RUN TestParse E0528 04:37:11.627177 29271 versions.go:96] unable to parse Kubernetes version "" E0528 04:37:11.627299 29271 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.031s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option --- PASS: TestBuildKubecfg (0.19s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option (0.01s) PASS ok k8s.io/kops/pkg/kubeconfig 0.264s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.117s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.076s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.084s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0528 04:37:11.687824 29280 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.076s === RUN Test_RunEtcdManagerBuilder I0528 04:37:11.707566 29285 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0528 04:37:11.757665 29285 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 W0528 04:37:11.759730 29285 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.154s === RUN Test_RunKubeApiserverBuilder I0528 04:37:11.697919 29288 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.095s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.112s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.037s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.067s === RUN TestServerGroupBuilder I0528 04:37:20.051474 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.072204 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.087398 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.106710 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.189856 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.215574 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.236719 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.250775 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.270106 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.336408 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.347039 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.357405 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.366008 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.376892 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.384777 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.392755 30154 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.35s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.441s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.030s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.012s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.027s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (2.71s) --- PASS: TestIssueCert/ca (1.15s) --- PASS: TestIssueCert/client (0.67s) --- PASS: TestIssueCert/clientOneYear (0.89s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 2.731s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.005s === RUN TestAddUntaggedRouteTables I0528 04:37:20.046911 30162 routetable.go:53] DescribeRouteTables: { } I0528 04:37:20.047061 30162 aws.go:229] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0528 04:37:20.047095 30162 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0528 04:37:20.047255 30162 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0528 04:37:20.047360 30162 aws.go:2059] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0528 04:37:20.047475 30162 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0528 04:37:20.047639 30162 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0528 04:37:20.047744 30162 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0528 04:37:20.047862 30162 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0528 04:37:20.048008 30162 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0528 04:37:20.048072 30162 volumes.go:128] DescribeVolumes: { } W0528 04:37:20.048108 30162 aws.go:2212] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0528 04:37:20.048328 30162 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:20.048448 30162 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:20.048514 30162 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:20.048609 30162 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:20.048667 30162 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:20.048751 30162 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:20.048815 30162 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:20.048848 30162 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:20.048929 30162 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:20.048977 30162 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:20.049028 30162 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0528 04:37:20.049077 30162 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0528 04:37:20.049113 30162 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0528 04:37:20.049177 30162 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0528 04:37:20.049218 30162 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0528 04:37:20.049283 30162 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0528 04:37:20.049360 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0528 04:37:20.049414 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0528 04:37:20.049453 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0528 04:37:20.049488 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0528 04:37:20.049541 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0528 04:37:20.049579 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0528 04:37:20.049623 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0528 04:37:20.049663 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0528 04:37:20.049704 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0528 04:37:20.049742 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0528 04:37:20.049783 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0528 04:37:20.049819 30162 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.092s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.031s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.069s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.008s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.005s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.004s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.005s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.039s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.085s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.006s === RUN TestRemovesDuplicateGuardedBlocks W0528 04:37:28.146183 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146243 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146257 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146270 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146282 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146295 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146307 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146320 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146332 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146344 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146356 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146371 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146382 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146395 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.146405 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.146418 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.164889 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.164936 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165040 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165061 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165157 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165177 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165269 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165288 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165376 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165395 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165490 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165509 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165608 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165621 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165701 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165713 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165769 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165780 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165843 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165855 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165925 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.165936 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.165990 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166001 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166064 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166081 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166151 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166164 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166229 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166247 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166340 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166358 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166421 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166437 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166524 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166544 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166618 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166636 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166772 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166790 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166877 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.166896 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.166983 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167000 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167079 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167097 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167182 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167198 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167277 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167303 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167388 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167405 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167483 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167503 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167581 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167598 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167674 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167690 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167785 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167809 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167893 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.167908 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.167990 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168004 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168082 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168097 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168175 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168189 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168270 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168287 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168384 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168400 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168495 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168511 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168600 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168616 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168692 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168708 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168778 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168793 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168887 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168902 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.168973 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.168988 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169059 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169074 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169148 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169163 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169229 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169245 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169314 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169329 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169397 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169412 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169493 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169516 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169592 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169606 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169678 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169693 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169749 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169760 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169814 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169825 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169886 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169897 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.169951 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.169962 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170015 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170026 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170080 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170091 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170144 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170154 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170208 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170218 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170273 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170286 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170340 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170351 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170403 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170414 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170481 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170493 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170569 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170580 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170638 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170649 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170701 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170711 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170766 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170776 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170846 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170857 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170910 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170921 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.170976 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.170986 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171039 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171050 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171102 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171112 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171167 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171177 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171230 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171240 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171308 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171319 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171373 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171384 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171436 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171447 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171501 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171512 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171564 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171574 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171648 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171659 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171714 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171725 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171779 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171789 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171842 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171853 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171907 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.171917 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.171993 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172003 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.172068 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172079 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.172131 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172142 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.172770 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172789 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.172866 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172881 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.172947 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.172961 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173030 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173045 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173112 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173127 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173192 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173207 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173272 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173286 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173352 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173367 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173433 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173448 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173516 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173530 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173595 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173610 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173677 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173691 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.173754 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.173769 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0528 04:37:28.174171 31489 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0528 04:37:28.174185 31489 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0528 04:37:28.174197 31489 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0528 04:37:28.174209 31489 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0528 04:37:28.174220 31489 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0528 04:37:28.174230 31489 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0528 04:37:28.174240 31489 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0528 04:37:28.174272 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174287 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174300 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.174315 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174328 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.174340 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174352 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.174363 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174375 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.174387 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174399 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174411 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174425 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174438 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.174448 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.174460 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.179995 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180034 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180133 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180170 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180260 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180275 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180358 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180372 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180509 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180526 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180607 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180625 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180717 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180736 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180824 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180841 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.180934 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.180952 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181054 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181072 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181204 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181223 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181323 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181342 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181447 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181465 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181591 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181611 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181705 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181724 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181819 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181840 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.181940 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.181959 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182058 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182076 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182198 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182219 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182329 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182347 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182443 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182461 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182550 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182572 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182694 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182714 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182809 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182829 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.182923 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.182943 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183038 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183056 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183138 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183157 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183268 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183289 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183418 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183439 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183535 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183556 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183652 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183673 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183768 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183789 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.183888 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.183909 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184015 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184035 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184133 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184154 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184255 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184275 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184371 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184391 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184484 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184507 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184608 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184627 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184721 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184741 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184830 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184849 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.184960 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.184982 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185073 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185093 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185191 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185213 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185308 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185328 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185422 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185442 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185538 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185559 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185649 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185667 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185764 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185783 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185874 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.185895 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.185986 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186007 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186108 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186128 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186250 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186270 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186399 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186419 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186510 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186529 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186618 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186637 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186724 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186747 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186843 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186864 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.186955 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.186975 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187084 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187105 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187196 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187216 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187315 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187336 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187432 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187451 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187546 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187567 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187662 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187683 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187777 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187798 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.187890 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.187911 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188026 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188046 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188144 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188165 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188269 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188294 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188392 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188412 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188504 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188522 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188616 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188637 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188735 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188756 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188855 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.188876 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.188982 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189003 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189139 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189160 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189253 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189274 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189373 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189397 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189495 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189516 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189617 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189637 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189754 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.189851 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.189989 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190012 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190114 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190181 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190316 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190343 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190444 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190464 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190556 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190615 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190752 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190775 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190868 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.190892 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.190994 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191014 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191108 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191128 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191229 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191249 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191345 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191366 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191467 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191487 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191593 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191613 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191708 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191728 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191822 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191842 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.191936 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.191956 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0528 04:37:28.192049 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0528 04:37:28.192069 31489 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.050s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.032s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.00s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.053s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.039s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0528 04:37:28.175290 31493 files.go:57] Writing file "/tmp/TestWriteFile2717300723/001/SubDir/test1.tmp" I0528 04:37:28.175410 31493 files.go:113] Changing file mode for "/tmp/TestWriteFile2717300723/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.044s === RUN TestBootstrapChannelBuilder_BuildTasks I0528 04:37:31.238607 32258 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:31.238761 32258 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:31.238812 32258 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:31.238861 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:31.238894 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:31.238946 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:31.238983 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:31.239000 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:31.239033 32258 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:31.239057 32258 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:31.248508 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.298141 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.452773 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.533566 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.623153 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.709146 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.797524 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:31.870973 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.019245 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.557827 32258 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.37s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0528 04:37:32.607699 32258 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:32.607782 32258 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:32.607830 32258 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:32.607863 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:32.607889 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:32.607921 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:32.607947 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:32.607964 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:32.607987 32258 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:32.608004 32258 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:32.608025 32258 featureflag.go:169] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0528 04:37:32.609564 32258 zones.go:93] ListHostedZonesPages { } I0528 04:37:32.725524 32258 featureflag.go:169] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.12s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0528 04:37:32.725665 32258 featureflag.go:166] FeatureFlag "ImageDigest"=false I0528 04:37:32.725736 32258 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:32.725784 32258 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0528 04:37:32.725818 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0528 04:37:32.725842 32258 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0528 04:37:32.725872 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0528 04:37:32.725898 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0528 04:37:32.725914 32258 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0528 04:37:32.725937 32258 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0528 04:37:32.725953 32258 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0528 04:37:32.729062 32258 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.09s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.08s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0528 04:37:32.923246 32258 new_cluster.go:1132] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestDefaultImage --- PASS: TestDefaultImage (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0528 04:37:32.958339 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.958399 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.958422 32258 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0528 04:37:32.958455 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0528 04:37:32.958471 32258 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0528 04:37:32.958511 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0528 04:37:32.958555 32258 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0528 04:37:32.958593 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.958611 32258 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0528 04:37:32.958640 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.958657 32258 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0528 04:37:32.958732 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.959150 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.959182 32258 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0528 04:37:32.959193 32258 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0528 04:37:32.959203 32258 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0528 04:37:32.959213 32258 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0528 04:37:32.959222 32258 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0528 04:37:32.959233 32258 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0528 04:37:32.959243 32258 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0528 04:37:32.959253 32258 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0528 04:37:32.960679 32258 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN TestKopsFeatureEnabled === RUN TestKopsFeatureEnabled/Missing_feature === RUN TestKopsFeatureEnabled/Existing_feature I0528 04:37:32.960904 32258 featureflag.go:166] FeatureFlag "Scaleway"=true --- PASS: TestKopsFeatureEnabled (0.00s) --- PASS: TestKopsFeatureEnabled/Missing_feature (0.00s) --- PASS: TestKopsFeatureEnabled/Existing_feature (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.823s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0528 04:37:31.207388 32257 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3988958937/001 I0528 04:37:31.207694 32257 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3988958937/001 I0528 04:37:31.207944 32257 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3988958937/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0528 04:37:31.212265 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0528 04:37:31.212334 32257 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0528 04:37:31.212396 32257 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0528 04:37:31.212523 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.212660 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.212731 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.212758 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.212945 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.212986 32257 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0528 04:37:31.213018 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.213086 32257 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0528 04:37:31.213170 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0528 04:37:31.213224 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.213269 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.213308 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.213322 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.213352 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.213383 32257 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0528 04:37:31.213432 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0528 04:37:31.213544 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.213591 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.213707 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.213753 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.213783 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.213805 32257 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0528 04:37:31.213881 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.214043 32257 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0528 04:37:31.214103 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.214128 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.214153 32257 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0528 04:37:31.214202 32257 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.214336 32257 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0528 04:37:31.214394 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.214417 32257 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0528 04:37:31.214452 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0528 04:37:31.214524 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.214562 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.214601 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.214615 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.214639 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.214667 32257 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0528 04:37:31.214703 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.214752 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.214774 32257 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0528 04:37:31.214805 32257 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.214848 32257 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0528 04:37:31.214878 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0528 04:37:31.214932 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0528 04:37:31.214982 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0528 04:37:31.215019 32257 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0528 04:37:31.215048 32257 internetgateways.go:66] CreateInternetGateway: { } I0528 04:37:31.215070 32257 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0528 04:37:31.215124 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.215165 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.215199 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.215212 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.215274 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.215297 32257 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0528 04:37:31.215324 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.215372 32257 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0528 04:37:31.215435 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0528 04:37:31.215494 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.215530 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.215565 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.215578 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.215606 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.215637 32257 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0528 04:37:31.215685 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0528 04:37:31.216123 32257 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender4057885580/001 I0528 04:37:31.216355 32257 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender4057885580/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0528 04:37:31.216699 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.216740 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.216820 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.216853 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.216874 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.216895 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.216946 32257 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0528 04:37:31.217130 32257 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0528 04:37:31.217190 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0528 04:37:31.217219 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0528 04:37:31.217273 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.217329 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.217372 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.217386 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.217410 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.217433 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.217468 32257 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0528 04:37:31.217531 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0528 04:37:31.217713 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.217749 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.217824 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.217856 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.217880 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.217900 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.217936 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.218036 32257 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0528 04:37:31.218073 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.218093 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.218113 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0528 04:37:31.218177 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.218214 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.218267 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.218284 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.218307 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.218328 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.218361 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.218411 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.218432 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0528 04:37:31.218543 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.218581 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.218655 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.218688 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.218707 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.218728 32257 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0528 04:37:31.218769 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.218878 32257 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0528 04:37:31.218924 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.218949 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.218966 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.219089 32257 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:31.219122 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0528 04:37:31.219189 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.219225 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.219263 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.219277 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.219299 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.219320 32257 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0528 04:37:31.219353 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.219408 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.219427 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.219469 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0528 04:37:31.219577 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.219613 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.219689 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.219722 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.219742 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.219763 32257 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0528 04:37:31.219795 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.219939 32257 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0528 04:37:31.219901 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.220023 32257 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0528 04:37:31.220073 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.220124 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.220177 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0528 04:37:31.220276 32257 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0528 04:37:31.220328 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.220373 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.220390 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.220419 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.220442 32257 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0528 04:37:31.220483 32257 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0528 04:37:31.220598 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.220653 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.220679 32257 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0528 04:37:31.220732 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0528 04:37:31.220786 32257 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0528 04:37:31.220850 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.220887 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.220923 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.220938 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.220998 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.221021 32257 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0528 04:37:31.221049 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.221082 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.221185 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0528 04:37:31.221210 32257 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0528 04:37:31.221238 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0528 04:37:31.221298 32257 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0528 04:37:31.221333 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.221366 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.221383 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.221413 32257 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0528 04:37:31.221444 32257 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0528 04:37:31.221495 32257 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0528 04:37:31.221571 32257 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:31.221623 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.221721 32257 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0528 04:37:31.221757 32257 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0528 04:37:31.221777 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.221798 32257 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:31.221831 32257 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:31.221866 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0528 04:37:31.221907 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.221921 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.221943 32257 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0528 04:37:31.221964 32257 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0528 04:37:31.222064 32257 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0528 04:37:31.222115 32257 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:31.222152 32257 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0528 04:37:31.222190 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0528 04:37:31.222205 32257 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0528 04:37:31.222232 32257 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.064s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.043s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.025s === RUN TestDiskRenderAzure I0528 04:37:31.452876 32382 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0528 04:37:31.453317 32382 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0528 04:37:31.453546 32382 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:124: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0528 04:37:31.454253 32382 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0528 04:37:31.454589 32382 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:102: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0528 04:37:31.454959 32382 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0528 04:37:31.455272 32382 resourcegroup.go:108] Creating a new Resource Group with name: rg I0528 04:37:31.455290 32382 resourcegroup.go:110] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0528 04:37:31.455556 32382 resourcegroup.go:108] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0528 04:37:31.456024 32382 subnet.go:107] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0528 04:37:31.456244 32382 virtualnetwork.go:122] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0528 04:37:31.456495 32382 virtualnetwork.go:122] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0528 04:37:31.456815 32382 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0528 04:37:31.457120 32382 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.031s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.037s === RUN TestProjectIAMBinding I0528 04:37:34.114881 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.115518 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.115546 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.116029 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.116058 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.116224 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0528 04:37:34.116361 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.116472 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.116493 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.116618 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.116641 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.116689 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0528 04:37:34.116795 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.117070 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.117093 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.117407 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0528 04:37:34.117436 33246 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0528 04:37:34.117579 33246 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.062s === RUN Test_OpenstackCloud_MakeCloud === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value --- PASS: Test_OpenstackCloud_MakeCloud (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value (0.00s) === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) === RUN Test_BuildClients === RUN Test_BuildClients/Empty_openstack_spec_means_no_load_balancer === RUN Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned === RUN Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned === RUN Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned --- PASS: Test_BuildClients (0.00s) --- PASS: Test_BuildClients/Empty_openstack_spec_means_no_load_balancer (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned (0.00s) === RUN Test_WaitForStatusActiveIsSuccessful --- PASS: Test_WaitForStatusActiveIsSuccessful (1.00s) === RUN Test_WaitForStatusActiveResultsInInstanceNotFound --- PASS: Test_WaitForStatusActiveResultsInInstanceNotFound (1.00s) === RUN Test_WaitForStatusActiveResultsInUnableToCreateServer --- PASS: Test_WaitForStatusActiveResultsInUnableToCreateServer (1.00s) === RUN Test_WaitForStatusActiveResultsInTimeout --- PASS: Test_WaitForStatusActiveResultsInTimeout (1.00s) === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 4.062s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.055s === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.030s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.006s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.031s === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0528 04:37:36.381986 33538 bindmount.go:223] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0528 04:37:36.382063 33538 bindmount.go:232] running mount command [mount -o remount,exec /containerized_mounter] I0528 04:37:36.382074 33538 bindmount.go:223] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0528 04:37:36.382087 33538 bindmount.go:243] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0528 04:37:36.382097 33538 bindmount.go:223] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0528 04:37:36.382107 33538 bindmount.go:223] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0528 04:37:36.382117 33538 bindmount.go:223] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0528 04:37:36.382590 33538 service.go:110] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.063s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.004s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.005s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.006s === RUN TestGetProxyEnvVars W0528 04:37:36.345555 33544 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.023s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.008s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.005s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.006s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.022s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0528 04:37:40.466164 33633 s3context.go:235] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.056s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 137.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.26.3-r0.apk >>> kops: Build complete at Sun, 28 May 2023 04:37:44 +0000 elapsed time 0h 3m 35s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/5) Purging .makedepends-kops (20230528.043411) (2/5) Purging kubernetes (1.27.2-r0) (3/5) Purging go (1.20.4-r0) (4/5) Purging bash (5.2.15-r5) Executing bash-5.2.15-r5.pre-deinstall (5/5) Purging readline (8.2.1-r2) Executing busybox-1.36.1-r0.trigger OK: 375 MiB in 103 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...